Solving A Trillion Polynomials

This is a long story: as in, it spans much of my life. This year, I solved a problem I’ve worked on since 2010.

University

It starts with my first year in university. I’d read Saladin’s “Anatomy & Physiology” until it was worn out, read through a medical dictionary cover to cover (predictable ending), and binged OpenCourseware series and lecturers on YouTube. But, I was intellectually green; heavy on facts and light on skill or understanding. I took my first mathematics classes, struggled, but loved them.

I started programming in R to save time. I hated hand-computing determinants and inverses of enormous matrices and had a suspicion programming would in some way be useful to me. I had fun. I scraped geospatial information from Twitter and plotted world maps, made dubious scientific charts, and played around. I blundered into unsolved problems like constructing Thomson spheres. I was learning, and had so much to learn.

I started solving polynomials. The setup was simple; I had a feeling that polynomial solutions would have an interesting pattern. I ended up with the algorithm:

• Use tuples drawn from a finite box of $\mathbb{Z}^n$ as polynomial coefficients
• Examine only monic polynomials (their largest order term has a coefficient of one)
• Construct a characteristic polynomial matrix
• Compute its complex eigenvalue solutions. I’d remembered the fundamental theorem of algebra from class; real solutions might not exist but complex ones would

I had a shallow notion of what any of this mathematics meant or how to compute these solutions. Despite this, I ran slow code on underpowered hardware computing a few polynomial solutions and got my graph.

It is, sadly, missing. I can find disturbing graphs of pre-melatonin sleep cycles, how many lines of code I changed, animations of numerically imprecise n-body simulations, and how many Google searches I made a day. But the original graphs of a handful of polynomial solutions are gone. Unfortunate, but: picture a few hundred dots in a circle. Close enough.

I knew I could eventually make a nicer graph. I decided to keep learning how to program and to revisit the problem when I was more skilled.

Attempt 1: 2016 - 2017

I kept studying biochemistry, but I found I had more aptitude for programming than lab-work. In 2014 I built a functional-programming library and started to understand abstraction, clean code, UX, performance, and how to program quickly and voluminously. I eventually noticed I was good and acquired a degree in software development to join the tech workforce. I could earn a living typing code (which I was doing in my free time anyway) and practise my craft. I was increasingly competent.

In 2016 I reapproached the polynomial problem and computed beautiful graphs of a billion polynomial solutions on a newer laptop.

The code is clean and legible, as good as anything I’ve ever written. It did take a few hundred commits to reach that state, and it wasn’t terribly fast. I had to be inventive to plot more solutions. For example, I pieced together the enormous final image from subtiles with ImageMagick; if I rendered in one pass, I’d reliably crash my laptop. This took time to discover (read: many crashes and eventual glee at discovering this hacky workaround).

The benchmarks left one trillion solutions flagged as new laptop 🙁

I made a few minor attempts following this one: in 2017, a larger Go version and a fairly quickly abandoned Rust version. They must not have worked well, as I can’t find renders I kept. Though, more inventiveness:

• Partitioning the coefficient space evenly across parallel workers
• Quantising solutions to a 64-bit number to save on storage (I calculated I’d need a terabyte of storage for the trillion-parameter run)

Attempt 2: 2020

By 2020, I’d written dozens more personal projects, and a lot of code professionally (though distressing amounts of PowerShell). I’d reached the peak of my ability to program. My 2020 attempt was faster; it stored solutions more efficiently:

• Discarded solutions quantised to the same spot on the Argand diagram
• Roots were quantised to 16-bit numbers
• Solution files were compressed
• It enumerated shells of coefficients to make resumption repeat less work.

Figuring out how to enumerate shells of coefficients stretched my programming abilities. The end result was single-threaded, but faster. In 2016, it took a day to compute a billion solutions. In 2020, thirty minutes.

A trillion solutions remained a distant possibility.

Attempt 3: 2026

I left my much beloved city to learn from the smartest people in Big Tech. I learnt so much, but rarely technical skills. I learnt how to distinguish what was required from what I preferred; what is needed and what is possible; how to compromise personal standards to deliver work quickly; how to spot the shortest path to a goal irrespective of my preferences. These things I have learnt well. I can build products quickly now.

On June 30th, I was probably bored. I don’t remember. This year, programming wasn’t fun, or really a verb; I could get Products and Features faster than ever and gain nothing in the process. Sometimes I would read, or sleep, or socialise; other times I would type my wishes and dreams into a TUI.

Bored, remembering a goal and knowing a fast path there, I probably typed:

“Clone poly2020, make it run on multiple processes”

into Claude. I[1] wrote it in JavaScript, but its unpleasant parallelism deterred me from using it. It probably responded with baffling gibberish that made my blood boil before rewriting my code to run on multiple processes. Then, I would have probably typed:

“Compute quartic roots directly”

knowing that fourth-order polynomials have closed-form solutions. It did, using Ferrari’s Method: a formula I do not understand from a mathematician I have not heard of. It involves a homomorphism from the symmetric group $S_4$ to $S_3$, with its kernel being its Klein four-group. I know a little about homomorphism and kernels and groups but not how it relates to polynomials. It might involve Galois Theory, but I haven’t made it past integers in Pinter’s “A Book of Abstract Algebra”.

I surely typed variations of “Be Faster” and “Be Better” several times; more experienced developers know that programs need to be made faster and made better several times. Less experienced developers forget this step. Many simply don’t care.

I ran my faster better program. It rendered images in exciting new palettes like “viridis” and “ice” and “inferno”. I hadn’t heard the term “viridis” before (it pertains to greenness)! Of course, with a trillion solutions, all fine structure is washed out and hidden. A smooth, glossy, featureless image.

Is There A Point?

I am honestly not sure. As I wait, millenarian, for gravity to remember those who spent thousands of millions of millions of dollars on perishable graphics cards, I guess this is a farewell to the craft of programming and a shrug at its output.

Certainly, I conveyed there was joy in practising a craft and testing my skill; the tests were meaningless but the practice and the craft were not. But: emphasis on the tense.

I am probably saying it’s better for your reach to continue to exceed your grasp. Also, fitting Christian allusions like “Journey Before Destination”[2], or: it’s unnatural to reap what was not sown.

The old world is dying, and the new world struggles to be born: now is the time of Idea Guys, Big Promises, Flim-Flam, Facades, Mirages, Black-Boxes, and Bafflegab.

Footnotes

[1] I meaning I. We need a new form of attribution meaning “I, but not me”

[2] Well, a Sanderson quote, but surely that counts.