YoBulk
Open Source Alternative To https://flatfile.com/

A Data exchange platform for all businesses that run on CSV files. Create YoBulk CSV Import button and make CSV importing 10x faster.

Website • Twitter • Slack

User Installation

• Docker

Docker

Option:1 Through Docker Compose

The docker image comes with a combined YoBulk backend image and MongoImage. We provide docker-compose.yml file under yobulkdev

git clone https://github.com/yobulkdev/yobulkdev.git
cd yobulkdev 
docker-compose up -d

• To make use of YoBulk Open AI feature,please make the below change.

Please open docker-compose.yml file and update the OPENAI_SECRET_KEY variable with the key generated from OpenAI API site.

The YoBulk dashboard is ready to go at http://localhost:5050/.

Option:2 Through Docker Run

If you are a Docker user, you may try this way!

• Prerequisites:You should have installed mongo on your local machine and it's running.

docker run --rm -it -p 5050:5050/tcp  yobulk/yobulk

• To make use of YoBulk Open AI feature,please make the below change.

Please pass the Open AI key generated from OpenAI API site in --env to explore YoBulk AI features.

Example:

docker run --rm -it -p 5050:5050/tcp  --env="OPENAI_SECRET_KEY=****" yobulk/yobulk

YoBulk dashboard is ready to go at http://localhost:5050/.

Building locally.

If you are a devevloper and want to build locally,you should have installed MongoDB and it should be running locally.

YoBulk is a fullstack next.js application which uses MongoDB as it's primary Database.

git clone https://github.com/yobulkdev/yobulkdev
cd yobulkdev
yarn install
yarn run dev

• To make use of YoBulk Open AI feature,please make the below change.

Please update OPENAI_SECRET_KEY variable in .env file with the Open AI key generated from OpenAI API site

The dashboard is ready to experiment at http://localhost:5050/.

Quick start

Please view the documentation to explore CSV importing flow.

Roadmap

• YoBulk Roadmap ReadMe

Features

Supported Features

•  No code Template and Import button creation
•  Smart auto matching between the CSV column and Template column
•  Custom validation rule setting against a field
•  Delightful Data review experience
•  Scalable through streaming.You can import a CSV with size of 1GB
•  GPT3 Integration
•  Template creation through JSON
•  Regex creation and addition in custom validation
•  Collaboration & workspace creation.

Demos

NoCode Template Creation Demo

All DataType Validation Demo

String Validation Demo

GPT Demo

Why are we building this?

More than 70% of the business data shared, is in the form of CSVs and Excel files. Less than 10% have truly adopted API integrations for data exchange.

As a developer and product manager, we have gone through the pain of building a scalable CSV importer.Everyday thousand of developers go through the same pain and hundreds of companies have to get spreadsheet and CSV data into a database. YoBulk is building an OSS standard for CSV and flatfile data onboarding where it can help you to build a data importing solution in-house.

Enterprises like Flatfile.com are addressing this problem but it has constraints like weak data securit and high pricing . While developers want an embeddable/scalable CSV importer for SaaS, the truth is that it is not very possible to come up with a one-size fits all solution in such a fragmented market and sector.

All these restrictions necessitate our mission - to make it possible for anyone to create an data import solution that suit their needs well.

To Developers, we would like to offer a bring your own validation rule and bring your own database experience. You can design and modify validations rules on your own way using YoBulk.

To enterprises , YoBulk is dedicated to bring data security. Moreover, you have 100% control of your data armed with long-term maintainability.

To be honest, we do not claim to outperform flatfile.com in terms of functionality and design, at least for now. Besides, our priority is to add more functionality at the moment. we would like to cultivate a community to democratise the knowledge and bring all CSV and flatfile open source projects under a single framework.

Read more Open vs Closed Data Onboarding

Our Mission

Our mission is to provide the most powerful flatfile(CSV,Spreadsheet)importer which is open source to every single internet business in the world. An AI first data exchange for businesses to share flatfiles where you are in-charge of your customer’s data onboarding, validation & transformation.

Support

If you have any questions or problems, please visit our GitHub discussions page, and we'll try to help you as soon as possible.

https://github.com/yobulkdev/yobulkdev/discussions

License

YoBulk is an open-source software with the AGPL 3.0 license.