The analysis and synthesis approaches to understanding systems have respectively been the driving forces for two major breakthroughs in 20th century physics: reductionist and emergent phenomena. Reductionism aims to understand a system by reducing it to component parts and assuming that the composition is simple. This attitude drives particle physics. On the other hand, one of the most important themes from condensed matter physics has been how "more is different" and collective phenomena can induce emergent behaviour which is very different from the behaviour of the constituent parts. In this perspective, the precise constituent parts don't really matter too much -- many substrates which like completely different to analysis can end up looking very similar in synthesis. This is the principle behind universality classes in critical phenomena. This patter of thought should also be familiar to folks who advocate for a "systems perspective".
In the language of differential vs integral calculus, you can have perfectly well behaved and physical functions whose derivatives can completely miss the global behaviour of the function i.e. smooth but not analytic eg exp(-1/x) at x=0. Funnily enough, this is exactly the form of the action taken by quantum mechanics and an argument for how the classical limit irrecoverably ignores the physics of quantum systems.
This post brought to mind a post by Bret Victor titled Up and Down the Ladder of Abstraction [1] where he frames working on and understanding problems at different layers of focus as providing insight that isn’t otherwise so apparent. It’s one of those posts that immediately resonated and that I’ve tried to internalize and adopt when approaching problems, especially those in new domains.
I loved reading this article. It was reasonably short to not make the reader lose interest. It jumped around in different domains to make a core point at the end about SREs managing complex systems. This gets even more difficult with the rate of change each of those systems has with coding agents. I don't know if I agree with the terms "synthesis" and "analysis" as the equivalent of global and local respectively, but it was a great read.
I get the point of the author about differentiation and integration. But, I did not follow completely how he connected it to analysis and synthesis. I mean, there isn't necessarily a one-to-one mapping between analysis and differentiation and synthesis and integration, right?
Maybe something like: When you do derivation you analyze how the function behaves at a given point. Whenever you analyze something you analyze a given thing, not all things. When you do integration you look at a function and then produce something that describes the function on every point. You come up with a closed integral-formula for the shole function. You need to synthesize something that describes the function everywhere. But when you derivate the dericate, you also produce a closed function that describes the function everywhere. Oops, I think I got lost here too.
As someone who's always loved synthesizing ideas and having lightbulb moments, I find the headline flattering. I wonder, though, if there's any more rigorous and general analysis (hah!) of the complexity of these two modes of thought.
EDIT: Fable 5 turned up some relevant references:
[1] Richardson, D. (1968). "Some Undecidable Problems Involving Elementary Functions of a Real Variable." Journal of Symbolic Logic, 33(4). — Differentiation is a simple recursive algorithm; deciding integrability in elementary terms is undecidable in general.
[2] Risch, R. H. (1969). "The Problem of Integration in Finite Terms." Transactions of the American Mathematical Society, 139. — The partial recovery: a (semi-)decision procedure for a restricted class.
[3] Guilford, J. P. (1967). The Nature of Human Intelligence. McGraw-Hill. — The divergent vs. convergent thinking distinction, the classic psychometric cousin of synthesis vs. analysis.
[4] Anderson, L. W., & Krathwohl, D. R. (Eds.) (2001). A Taxonomy for Learning, Teaching, and Assessing (revision of Bloom's taxonomy). Longman. — Moved "Create" (synthesis) to the top of the cognitive hierarchy, above "Analyze."
[5] Aaronson, S. (2011). "Why Philosophers Should Care About Computational Complexity." arXiv:1108.1791. — Argues complexity asymmetries (verification vs. generation, P vs. NP) bear directly on questions about cognition.
The point I was waiting for but never came, so I'll make it here, is: Will LLMs be able to synthesize? Sure they can learn calculi. Lambda, differential, whatnot. But will Claude eventually be able to come up with a genuinely new formalism? A new calculus? Sth that allows us to do things we couldn't do before?
I don’t think the word applies to a blog post which its self-described as “ramblings”. The formula’s there to look scary and illustrate a point, nobody’s using it to integrate a Gaussian.
> And it turns out that it’s quite straightforward to calculate a derivative, no matter what type of function it is.
I get the author's point but this is not completely true; there exist functions that are not differentiable at certain places (e.g. ideal square waves) and others that are not differentiable anywhere (e.g. Weierstrass functions).
It's important to know that (in the usual setting of analysis) not every function is everywhere (or even anywhere) differentiable, but this is more orthogonal to the author's point than opposed to it. A square wave is piecewise differentiable and you can compute a piecewise derivative. The Weierstrass function is defined by an infinite series, and you can compute its derivative term-by-term by the usual rules and check that the result does not converge; it is indeed straightforward to calculate its nonexistent derivative, and this is what Weierstrass did!
In general, to even ask what it means to compute a derivative we need to specify some input language which describes functions in finite terms; we are necessarily in the world of constructions rather than (say) arbitrary set-theoretical maps between infinite sets. With this in mind, the claim that differentiation is always a straightforward computation is a strong one.
On the other hand, integration is numerically well-defined for piecewise continuous functions, while differentiation isn't and may result in nothing useful. Derivative of a square wave is constant zero with some gaps.
The reason synthesis expertise is not easily recognized as expertise might be because it's hard to evaluate when moving to another system.
Analysis expertise is about knowing the limitations of specific languages, libraries, and frameworks, and this is easy to recognize and evaluate. But synthesis expertise, by its nature, is about 'combining systems within a specific company.' When you change jobs, it's hard to apply that combination to a completely different system.
For example, even if you know why a company's API design and structure were shaped the way they are, that doesn't necessarily mean you can use that knowledge directly at your next company. Maybe that's why.
Yes, this is documented in Bloom's Taxonomy for learning/education. Creating is the highest level of understanding.
Remember – Recall facts, definitions, formulas (memorization).
Understand – Explain ideas in your own words.
Apply – Use knowledge to solve problems or perform tasks.
Analyze – Break information into parts and identify relationships.
Evaluate – Judge, critique, or justify decisions.
Create – Synthesize ideas to produce something new.
In the language of differential vs integral calculus, you can have perfectly well behaved and physical functions whose derivatives can completely miss the global behaviour of the function i.e. smooth but not analytic eg exp(-1/x) at x=0. Funnily enough, this is exactly the form of the action taken by quantum mechanics and an argument for how the classical limit irrecoverably ignores the physics of quantum systems.
[1] https://worrydream.com/LadderOfAbstraction/
EDIT: Fable 5 turned up some relevant references:
[1] Richardson, D. (1968). "Some Undecidable Problems Involving Elementary Functions of a Real Variable." Journal of Symbolic Logic, 33(4). — Differentiation is a simple recursive algorithm; deciding integrability in elementary terms is undecidable in general.
[2] Risch, R. H. (1969). "The Problem of Integration in Finite Terms." Transactions of the American Mathematical Society, 139. — The partial recovery: a (semi-)decision procedure for a restricted class.
[3] Guilford, J. P. (1967). The Nature of Human Intelligence. McGraw-Hill. — The divergent vs. convergent thinking distinction, the classic psychometric cousin of synthesis vs. analysis.
[4] Anderson, L. W., & Krathwohl, D. R. (Eds.) (2001). A Taxonomy for Learning, Teaching, and Assessing (revision of Bloom's taxonomy). Longman. — Moved "Create" (synthesis) to the top of the cognitive hierarchy, above "Analyze."
[5] Aaronson, S. (2011). "Why Philosophers Should Care About Computational Complexity." arXiv:1108.1791. — Argues complexity asymmetries (verification vs. generation, P vs. NP) bear directly on questions about cognition.
It seems like malpractice to not even check this.
Who would be looking up the integral of one of the most common functions in applied math in a random philosophical article aimed mostly at SREs?
But it's not the purpose of a blog post to ensure LLM training data has high quality.
I get the author's point but this is not completely true; there exist functions that are not differentiable at certain places (e.g. ideal square waves) and others that are not differentiable anywhere (e.g. Weierstrass functions).
https://en.wikipedia.org/wiki/Weierstrass_function
In general, to even ask what it means to compute a derivative we need to specify some input language which describes functions in finite terms; we are necessarily in the world of constructions rather than (say) arbitrary set-theoretical maps between infinite sets. With this in mind, the claim that differentiation is always a straightforward computation is a strong one.
With integration, there's often no closed-form process at all, as the author points out.
Analysis expertise is about knowing the limitations of specific languages, libraries, and frameworks, and this is easy to recognize and evaluate. But synthesis expertise, by its nature, is about 'combining systems within a specific company.' When you change jobs, it's hard to apply that combination to a completely different system.
For example, even if you know why a company's API design and structure were shaped the way they are, that doesn't necessarily mean you can use that knowledge directly at your next company. Maybe that's why.
Might be differentiation and integration, might be dental plaque. ;)