I built aimake — an incremental build system for AI/ML pipelines, basically make for AI apps. GitHub: https://github.com/arjun988/aimake The idea is simple: Dataset → Preprocess → Embeddings → Index → Prompt → Eval → Report aimake builds a dependency graph, fingerprints inputs by content rather than timestamps , and only rebuilds steps whose inputs actually changed. So if you change your prompt: Before: Dataset ✓ Preprocess ✓ Embeddings ✓ Index ✓ Prompt ✗ changed Eval ✗ Report ✗ After: 2 rebuilt · 5 reused aimake plan # see what would rebuild aimake build # only stale steps run aimake explain # see why a step needs rebuilding It’s not Airflow (orchestration) and it’s not DVC (data versioning alone). It’s make for AI pipelines: dependency graph + content fingerprints + incremental builds + caching. Shipped so far Content-hash fingerprints instead of mtime-based caching Incremental + parallel builds plan / build / explain CLI Experiment comparison + hyperparameter search S3 cache + Hugging Face / DVC / Docker / Ollama / W&B plugins You can try it with: pip install aimake The main reason I built this is that AI pipelines are expensive to rerun. Changing a prompt shouldn’t mean recomputing your dataset, embeddings, vector index, etc. If you build RAG , evaluation or any AI pipelines, I’d love feedback: what’s the most painful step you wish was cached? And if you think the idea is useful, a ⭐ on GitHub would really help. submitted by /u/Miserable_Extent8845
Originally posted by u/Miserable_Extent8845 on r/ArtificialInteligence