Rapid Lightning Genetic-programming ensembles for tabular classification. Competes or beats (recently outdated) gradient-boosted decision trees (GBDTs) on tabular classification. Evolved small algebraic programs combined through a linear head, then the whole model is compiled to a dependency-free C .so for tens-of-nanoseconds inference. Although foundation models like the infamous TabPFN have taken the stage for inference, there yet remains many places for these ultrafast and tiny decision makers that can run on commodity CPU. A novel method, a full compile-to-C toolchain, and a rigorous benchmark showing it does not beat tuned gradient-boosted trees, even after months of trying really, really hard. But it’s still pretty darn cool and exposes some cool methods. First, the three-month story Ah the memories… I first tried Claude Code three months ago. Immediately I saw the opportunity to play with genetic programming, evolutionary algorithms, and all kinds of weird stuff that I had never had the time or been a good enough coder to play with. And I got the first taste of what it’s like DELVING deep into places you have only the most basic understanding of. I studied some machine learning, and this is a deep, deep place. Genetic programming and evolution were deeper than I expected by far I don’t need to tell you all about the wild Dunning-Kruger roller coaster ride it is to sit in the copilot seat with a hyperintelligent machine that constantly thinks you’ve made a machine learning breakthrough because it thinks you’re in 2016. I don’t need to tell you fellows what it’s like having to constantly remind said intelligent entity that yes, sub-millisecond inference isn’t groundbreaking, everybody does it now, please search the web AGAIN. And all of you are certainly familiar with the reply “…and it’s deeper that I first indicated…” so called insights/apologies from our favorite robot. Yet through it all, with rigor, even a crusty old technologist can get something real and actual. If you push hard and be your own hardest critic, you can make something neat. Evolution is slow but amazing We (Claude and I) tried two objectives: v1, where members are evolved as predictors (accuracy + AdaBoost-style boosting), and v3 “head-aware”, where members are evolved as signal generators for the linear head. The head-aware won, and it was a trip. Read the notebooks for more info. It was a real ‘evolution take the wheel’ moment when I suggested the method. I wasn’t overly surprised to learn head-aware was a thing, so this method was a re-invention. I still felt pretty smart though. A fast horse in the age of the car It makes sense that the farthest an AI can take you is to the end of its training data. We’re so early in this vibe coding that when you present the code you’ve been working on to a fresh context, Claude will praise you for what clean code you’ve written! The coding AI aren’t even aware of coding AI yet. And yet, if you have the fortitude, are rigorous and critical, and make sure to make sure you are not fooling yourself (and you are the easiest person for you to fool) it is possible to push the edge of the envelope. I have made a weird monster alien method here. It evolves ensemble member trees that individually don’t even make predictions (barely better than random), yet each tree has been selected over millions of rounds for the unique ‘signal’ it generates for the ‘head’ - a logistic regression method that simply takes all the ensembles’ signals and combines them for an output prediction. And for some reason (which Claude or a true machine learning scientist perhaps like yourself can explain) it works better having a bunch of bad predictors tell a smart head what they think, versus a bunch of smart predictors telling the head. Knowledge or curiosity? I was always interested in genetic programming and have a fair familiarity with machine learning, but let me tell you, I was not prepared for the depth of the fields (dunning kruger strikes again!). GP, although largely abandoned (except for syzkaller or other fuzzers and some design work) is a rich field with a lot of room still remaining for research, but it is deep. And the machine learning field, though quieter as of late, is as deep as computer science itself. I waded way far out into these fields. Send help. At the end of this, I have learned a lot. But what I learned most of all is that you have to test your knowledge. Curiosity brings you to the start of the journey, but knowledge waits at the end. If you can make it. You have to TEST what you made. Benchmark. Make sure. And probably most importantly, if you can help it, try to actually KNOW something about what you are working on. Better yet, if you can manage it, try to work with an ACTUAL EXPERT IN THE FIELD - you’ll get better results! Hence my post here. And so, I drop here with the good old Apache 2.0 license (because that was suggested), Rapid Lightning, my three months of work, with the hope that you find an application, or that you can glean something from the cool genetic programming methods I employed and augmented (the symbolic regression explorations into algebraically invertible genomes was especially heady, and very interesting, though admittedly a bit of a re-exploration). Most everything is in Jupyter notebooks intended to run on Google Colab (most run on free tier without GPU needed) or simple Python. Please, if you find this useful or interesting, let me know! And if you happen to discover some cool science of your own, especially any shortcuts to evolution, let us know! Happy vibing and research deathcloset/RapidLightning submitted by /u/powerscunner
Originally posted by u/powerscunner on r/ArtificialInteligence