MyGit

nucleuscloud/neosync

Fork: 120 Star: 3370 (更新于 2024-11-14 08:39:10)

license: NOASSERTION

Language: Go .

Open source data anonymization and synthetic data orchestration for developers. Create high fidelity synthetic data and sync it across your environments.

最后发布版本: v0.4.80 ( 2024-10-23 01:47:41)

官方网址 GitHub网址

Open Source Data Anonymization and Synthetic Data Orchestration

| Website | Docs | Discord | Blog | Changelog | Roadmap

PRs Welcome Go Tests Follow X ArtifactHub Neosync

Introduction

Neosync is an open-source, developer-first way to anonymize PII, generate synthetic data and sync environments for better testing, debugging and developer experience.

Companies use Neosync to:

  1. Safely test code against production data - Anonymize sensitive production data in order to safely use it locally for a better testing and developer experience
  2. Easily reproduce production bugs locally - Anonymize and subset production data to get a safe, representative data set that you can use to locally reproduce production bugs quickly and efficiently
  3. High quality data for lower-level environments - Catch bugs before they hit production when you hydrate your staging and QA environments with production-like data
  4. Solve GDPR, DPDP, FERPA, HIPAA and more - Use anonymized and synthetic data to reduce your compliance scope and easily comply with laws like HIPAA, GDPR, and DPDP
  5. Seed development databases - Easily seed development databases with synthetic data for unit testing, demos and more

Features

  • Generate synthetic data based on your schema
  • Anonymize existing production-data for a better developer experience
  • Subset your production database for local and CI testing using any SQL query
  • Complete async pipeline that automatically handles job retries, failures and playback using an event-sourcing model
  • Referential integrity for your data automatically
  • Declarative, GitOps based configs as a step in your CI pipeline to hydrate your CI DB
  • Pre-built data transformers for all major data types
  • Custom data transformers using javascript or LLMs
  • Pre-built integrations with Postgres, Mysql, S3

Getting started

Neosync is a fully dockerized setup which makes it easy to get up and running.

A compose.yml file at the root contains production image refs that allow you to get up and running with just a few commands without having to build anything on your system.

Neosync uses the newer docker compose command, so be sure to have that installed on your machine.

To start Neosync, clone the repo into a local directory, be sure to have docker installed and running, and then run:

make compose/up

To stop, run:

make compose/down

Neosync will now be available on http://localhost:3000.

The production compose pre-seeds with connections and jobs to get you started! Simply run the generate and sync job to watch Neosync in action!

Kubernetes, Auth Mode and more

For more in-depth details on environment variables, Kubernetes deployments, and a production-ready guide, check out the Deploy Neosync section of our Docs.

Resources

Some resources to help you along the way:

Contributing

We love contributions big and small. Here are just a few ways that you can contribute to Neosync.

Licensing

We strongly believe in free and open source software and make this repo is available under the MIT expat license.

最近版本更新:(数据更新于 2024-10-23 13:06:13)

2024-10-23 01:47:41 v0.4.80

2024-10-16 10:38:58 v0.4.79

2024-10-16 02:44:54 v0.4.78

2024-10-10 02:28:39 v0.4.77

2024-10-09 02:13:00 v0.4.76

2024-10-08 03:44:13 v0.4.75

2024-10-04 01:41:17 v0.4.74

2024-10-03 06:53:05 v0.4.73

2024-10-02 01:34:36 v0.4.72

2024-09-27 08:18:56 v0.4.71

主题(topics):

benthos, docker, etl, faker, fine-tuning, golang, kubernetes, mysql, nextjs, open-source, orchestration, postgresql, reactjs, self-hosted, synthetic-data, synthetic-data-generation, test-data-generator, testing, typescript

nucleuscloud/neosync同语言 Go最近更新仓库

2024-11-21 22:49:20 containerd/containerd

2024-11-21 13:50:50 XTLS/Xray-core

2024-11-21 07:36:18 kubernetes/kubernetes

2024-11-21 06:27:30 ollama/ollama

2024-11-21 05:17:55 Melkeydev/go-blueprint

2024-11-21 04:04:03 dolthub/dolt