Hey everyone, I just shipped aimake 2.0 , an open-source incremental build system for AI/ML pipelines. The idea is pretty simple: if your pipeline looks something like data → preprocessing → model → embeddings → prompt → eval → report and you change one thing , you shouldn’t have to rerun everything. aimake builds a dependency graph and uses SHA-256 content fingerprints to figure out what actually changed, then reuses everything else from cache. So instead of: change one prompt → rerun entire pipeline → waste compute + API calls you get: change one prompt → aimake plan → rebuild only affected steps → reuse everything else What’s in 2.0 Incremental + parallel builds Content-addressed caching plan , build , and explain CLI Cost + token estimation before running Interactive TUI and web dashboard Python + TypeScript SDKs Shared S3/remote cache Experiment comparison + hyperparameter search Lineage and reproducibility tracking Plugins for Hugging Face, W&B, DVC, Docker and Ollama Official Docker image for CI It’s basically make for AI applications — dependency tracking and caching, but for the things AI pipelines actually deal with: datasets, models, prompts, embeddings, evaluations and generated artifacts. Quick start: pip install aimake aimake init aimake plan aimake build GitHub: https://github.com/arjun988/aimake Docs: https://aimake-doc.vercel.app/ PyPI: https://pypi.org/project/aimake/ I built this because I got tired of changing one small thing in an AI pipeline and watching everything run again. If you work on AI/ML pipelines, give it a try. And if you think the idea is useful, a GitHub ⭐ would mean a lot and help me get the project in front of more people. I’d also genuinely like to know: what part of your AI pipeline wastes the most time when you have to rerun it? submitted by /u/Miserable_Extent8845
Originally posted by u/Miserable_Extent8845 on r/ArtificialInteligence