nucleuscloud/neosync
Fork: 120 Star: 3370 (更新于 2024-11-14 08:39:10)
license: NOASSERTION
Language: Go .
Open source data anonymization and synthetic data orchestration for developers. Create high fidelity synthetic data and sync it across your environments.
最后发布版本: v0.4.80 ( 2024-10-23 01:47:41)
Open Source Data Anonymization and Synthetic Data Orchestration
Introduction
Neosync is an open-source, developer-first way to anonymize PII, generate synthetic data and sync environments for better testing, debugging and developer experience.
Companies use Neosync to:
- Safely test code against production data - Anonymize sensitive production data in order to safely use it locally for a better testing and developer experience
- Easily reproduce production bugs locally - Anonymize and subset production data to get a safe, representative data set that you can use to locally reproduce production bugs quickly and efficiently
- High quality data for lower-level environments - Catch bugs before they hit production when you hydrate your staging and QA environments with production-like data
- Solve GDPR, DPDP, FERPA, HIPAA and more - Use anonymized and synthetic data to reduce your compliance scope and easily comply with laws like HIPAA, GDPR, and DPDP
- Seed development databases - Easily seed development databases with synthetic data for unit testing, demos and more
Features
- Generate synthetic data based on your schema
- Anonymize existing production-data for a better developer experience
- Subset your production database for local and CI testing using any SQL query
- Complete async pipeline that automatically handles job retries, failures and playback using an event-sourcing model
- Referential integrity for your data automatically
- Declarative, GitOps based configs as a step in your CI pipeline to hydrate your CI DB
- Pre-built data transformers for all major data types
- Custom data transformers using javascript or LLMs
- Pre-built integrations with Postgres, Mysql, S3
Getting started
Neosync is a fully dockerized setup which makes it easy to get up and running.
A compose.yml file at the root contains production image refs that allow you to get up and running with just a few commands without having to build anything on your system.
Neosync uses the newer docker compose
command, so be sure to have that installed on your machine.
To start Neosync, clone the repo into a local directory, be sure to have docker installed and running, and then run:
make compose/up
To stop, run:
make compose/down
Neosync will now be available on http://localhost:3000.
The production compose pre-seeds with connections and jobs to get you started! Simply run the generate and sync job to watch Neosync in action!
Kubernetes, Auth Mode and more
For more in-depth details on environment variables, Kubernetes deployments, and a production-ready guide, check out the Deploy Neosync section of our Docs.
Resources
Some resources to help you along the way:
- Docs for comprehensive documentation and guides
- Discord for discussion with the community and Neosync team
- (https://x.com/neosynccloud) for the latest updates
Contributing
We love contributions big and small. Here are just a few ways that you can contribute to Neosync.
- Join our Discord channel and ask us any questions there
- Open a PR (see our instructions on developing with Neosync locally)
- Submit a feature request or bug report
Licensing
We strongly believe in free and open source software and make this repo is available under the MIT expat license.
最近版本更新:(数据更新于 2024-10-23 13:06:13)
2024-10-23 01:47:41 v0.4.80
2024-10-16 10:38:58 v0.4.79
2024-10-16 02:44:54 v0.4.78
2024-10-10 02:28:39 v0.4.77
2024-10-09 02:13:00 v0.4.76
2024-10-08 03:44:13 v0.4.75
2024-10-04 01:41:17 v0.4.74
2024-10-03 06:53:05 v0.4.73
2024-10-02 01:34:36 v0.4.72
2024-09-27 08:18:56 v0.4.71
主题(topics):
benthos, docker, etl, faker, fine-tuning, golang, kubernetes, mysql, nextjs, open-source, orchestration, postgresql, reactjs, self-hosted, synthetic-data, synthetic-data-generation, test-data-generator, testing, typescript
nucleuscloud/neosync同语言 Go最近更新仓库
2024-11-21 22:49:20 containerd/containerd
2024-11-21 13:50:50 XTLS/Xray-core
2024-11-21 07:36:18 kubernetes/kubernetes
2024-11-21 06:27:30 ollama/ollama
2024-11-21 05:17:55 Melkeydev/go-blueprint
2024-11-21 04:04:03 dolthub/dolt