Today, AI coding tools are lowering the barrier to programming at an unprecedented pace.

More and more people are building products with their own hands, and revisiting a fundamental question: what kind of craft is programming? Why is it so captivating, and why is it so hard?

Frederick P. Brooks Jr.'s "The Tar Pit," written fifty years ago in The Mythical Man-Month, reads with surprising relevance today.

Brooks is best known for leading the development of IBM's System/360 and its operating systems. He likened building large software systems to prehistoric beasts trapped in tar pits: the real danger was never any single difficulty, but countless problems surfacing simultaneously and entangling with one another. The harder a team struggled, the deeper it sometimes sank.

Programming is both painful and joyful. It mires you in trivialities, yet also empowers creators to turn ideas into reality.

May you pass through the tar pit, and still find joy in creation.

The Tar Pit

In prehistoric times, one of the most vivid scenes was that of great beasts struggling to their death in tar pits.

Dinosaurs, mammoths, saber-toothed tigers... the harder they fought to break free, the tighter the tar gripped them. No beast was powerful enough to avoid sinking.

For the past decade, large-system programming has been the tar pit.

Many a powerful team has thrashed violently in it. Many have emerged with running systems — few have met goals, schedules, and budgets simultaneously.

The difficulties are not inherent to any single aspect. Any one problem seems solvable, like a single paw that can still pull free. What is truly terrifying is that many factors appear simultaneously and interact with each other, ultimately pulling the entire project deeper and deeper.

To solve this, one must first understand the craft of programming itself, and the intrinsic joys and woes within it.

Program, Product, System

We often read stories in the news of two programmers in a converted garage whose program outperformed a giant's.

Every programmer wants to believe such stories, because we know that if it's just writing a program, our speed far exceeds an organization's "1000 lines per year" pace.

So why haven't big-company development teams been replaced by focused garage duos?

In 1976, 21-year-old Steve Jobs and his friend Steve Wozniak founded Apple Computer in a garage

A "program" is something complete in itself, runnable by its author in the development environment. This is the garage project, and what programmers implicitly reference when estimating their own productivity.

But for a program to become truly useful, it must cross two boundaries.

The first boundary is from program to product.

It no longer suffices for the author alone to run it. Anyone must be able to run, test, repair, and extend it. It must function across different operating environments and handle diverse data.

To become a general product, the program must be written in a more generalized way. Especially the range and form of inputs must be generalized as far as the basic algorithm reasonably permits.

Then the program must be thoroughly tested, so that one may trust it. This means preparing, running, and documenting a substantial body of test cases to explore its input range and boundaries. Finally, complete documentation is needed so anyone can use, repair, and extend it.

By experience, a product costs at least three times as much as a debugged program with the same function.

The second boundary is from program to component in a system.

A system is not an isolated program but a collection of interacting programs, coordinated in function and constrained in format.

To become a system component, every input and output of the program must conform in format and meaning to defined interface specifications. It must also be designed to a predetermined resource budget — memory space, I/O devices, compute time. Finally, it must be tested with other components in all expected combinations.

This testing must be extensive, because the number of test cases grows combinatorially. It is also time-consuming. Even if each component is debugged in isolation, unexpected interactions between components can still produce subtle bugs.

A system component also costs at least three times as much as a standalone program with the same function.

When a program is simultaneously a product and a component, its cost is nine times that of a simple program. Yet this is the truly useful result, and the goal of most development efforts.

The Joys of the Craft

Why is programming fun?

First is the sheer joy of making things.

As a child delights in his mud pie, so the adult enjoys building things, especially things of his own design.

I think this joy must be an echo of God's delight in creating — each leaf, each snowflake unique and fresh.

Second is the pleasure of making things that are useful to others.

Deep down, we want others to use our work and find it helpful. In this, a programming system is no different from a child's first clay pencil holder "for Daddy's office."

Third is the fascination of fashioning complex puzzle-like objects.

A program is like a puzzle of interlocking parts, watching them mesh in cycles as the rules and logic written into it unfold step by step.

A properly programmed computer is like a complex pinball machine or jukebox: you press buttons, mechanisms engage inside, and out comes a result.

Except programming pushes this fascination to the extreme.

ENIAC, one of the world's earliest general-purpose electronic computers

Fourth is the joy of always learning.

Programming tasks are rarely repetitive; problems keep presenting themselves in new ways. The solver always learns something — sometimes practical, sometimes theoretical, sometimes both.

Finally, there is the joy of working in a tractable medium.

Like the poet, the programmer works only with pure thought-stuff. He builds his castles in the air, creating new worlds from imagination alone.

Few creative media are so flexible, so easy to polish and rework and restructure, so well-suited to realizing grand conceptual visions.

Yet a program differs from a poet's words. It is real in a sense, for it moves, it works, it acts upon the real world. It prints results, draws pictures, produces sounds, moves mechanical arms.

The magic of myths and legends has become reality in our time. People type the right incantations on keyboards, and the screen comes alive, showing things that never were and could not be.

Programming is fun because it gratifies creative longings built deep within us, and taps into sensibilities we all share.

The Woes of the Craft

Of course, programming is not all joy.

Understanding its intrinsic woes makes them easier to bear when they come.

The first woe is the necessity for perfection.

The computer resembles magic in this too: one wrong character, one misplaced pause, and the spell fails.

Humans are not accustomed to perfection. Few activities demand it. I think adapting to this requirement of perfection is the most difficult part of learning to program.

The second woe is that responsibility often exceeds authority.

Your goals are set by others; information and resources are provided by others. The programmer rarely controls his working conditions, sometimes not even his objectives. In management terms, "one's authority is not commensurate with his responsibility."

But this seems true of all fields.

Work that truly gets things done almost never comes with formal authority matching its responsibility. In practice, real authority comes not from titles but from influence built through consistently delivering results.

VisiCalc, launched in 1979, was the world's first commercially successful standalone software and the first electronic spreadsheet

For many programmers, there is a particular woe: dependence on other people's programs.

These are often poorly designed, crudely implemented, incompletely delivered — no source code, no test cases, poor documentation. So he must spend much time fixing what in an ideal world would be complete, accessible, and usable.

Then there are more trivial woes.

Designing grand architectures is interesting, but hunting small bugs is also part of the work. No creative activity is free of drudgery, tedium, and painstaking labor; programming is no exception.

Further on, you find that debugging does not converge rapidly toward an endpoint as you might hope. It proceeds slowly, growing harder toward the end. Testing keeps getting pushed back; the last few stubborn bugs take more time than all the initial ones combined.

The last woe, sometimes the final straw, is that the product you toiled over for so long is already obsolete when finished — or before.

Your peers are hotly pursuing newer, better ideas. Your labor of love has not only been conceived but already scheduled.

But when you complete your product, that newer, better thing has not actually been built — it exists only in discussion. It too will take months to develop.

Until it is actually put to use, real products are never as glamorous as paper proposals; yet once real problems must be solved, real products have irreplaceable value and satisfaction.

Technology always moves forward.

Conceptually, once a design is finalized, it is already obsolete. But real products take time to land, and demand trade-offs. Whether a solution is obsolete should be judged against what already exists, not against ideas not yet realized.

The true challenge and mission is to find real solutions to real problems, within given time, with available resources.

This is programming. It is the tar pit into which countless efforts have sunk, and it is also the joy felt through creating value.

Text by Yanyang Edited by Cindy