We Were Promised Flying Cars and Five Hour Work Weeks

Work is not a zero-sum game

Image courtesy of Clem Onojeghuo

Remember those stories from a few decades ago, about how improvements in productivity and automation would mean that we’d only be working 5 hours a week by now? They seem as fanciful, and predictive, as The Jetsons.

It’s almost a punch-line now, because many actually work more hours today than back then. How could they have been so stupid?

Of course there is actually a lot baked into the fact that higher productivity has not reduced hours worked (nor, it turns out, has it raised wages for the bottom 50% of earners in the last three decades, quite contrary to the previous, oh, 200 years). But a big part of why that prediction seems so silly now is it included an implicit assumption: That the work we’d be doing today is pretty much the same as what we did then. You know, before computers.

And I don’t mean, they could not foresee how much time we’d waste on social media. Their primary mistake was not thinking about how we’d use the space created by greater productivity.

We’ve known for centuries that increased efficiency results in increased consumption — we keep improving fuel efficiency, yet we’re consuming more gas than ever. Why?

We also know that expanding a freeway has never reduced traffic congestion. It has the same cause.

Induced demand. People who would not have driven with only two lanes, because of all the traffic, will now drive if there are four lanes. Trips you would not have taken because of the gas cost now become affordable and reasonable.

If your goal was to get more people driving, and maybe get the resulting increase in economic activity, then I suppose you got what you wanted. But if you were trying to reduce traffic — which is why most people say they want more freeways — then you’ve failed. If anything, you made it worse, because now twice the number of people are stuck in traffic. You didn’t reduce the problem, you just spread it around.

Demand is induced in many other areas. I grew up remodeling houses with my dad and got to see the impact of pneumatic nail guns and paint sprayers, which automated a huge chunk of how we’d spent our days. Of course, we didn’t use that newly free time to sit around, we changed our behavior: We did higher quality work, we finished jobs faster so got more done in a year, and we lowered prices.

I ran into this perception conflict all the time as I was building Puppet. I’d say I was building automation for the data center, and salespeople and execs would say, “Oh, so you can fire sysadmins!” I don’t know what they had against operations teams, but no, I would respond: I can give you a choice between reduced cost at the same service level (i.e., you fire people), or keep costs the same but increase service quality.

“Wait, that’s an option?!” Without higher level tools like Puppet, people had no idea how to increase service quality. Cost was their only dial, so they focused on that, even if it made no sense. Once I gave them other choices, of course they wanted better quality.

So where success in 1999 was shipping twice a year, and not going down, like, that often, now success is shipping multiple times a day and never appearing down to your customers. The standards have changed entirely, and that change was made possible mostly through automation. You can bet Netflix doesn’t become the new standard for infrastructure hotness using the bad old manual practices of the 90s.

You might say, no, people’s standards changed and that’s what motivated them to invest in automation, but you would be wrong. They always wanted higher quality. They always wanted better service. They just felt they had no choice but to accept the status quo, until we showed them other options.

I’m not saying automation never destroys jobs, never reduces the amount of work to be done in an area. But it’s by no means the default result.

The first impact of automation is to increase quality. I felt this myself, building houses. The main reason I’m no longer a carpenter is because of how incompetent I was at setting trim nails. You drive the nail most of the way into the wood1, then use a small punch to recess it. You cover that one little hole in spackle, and no one can tell. Unless you’re me. When I do it, I make a little sunflower, with a hole in the middle and holes all around in a circle, because I’m just that good at letting the punch head slide off the nail, into the wood. If we’d had the trim nailers that exist now, even I could have done a decent job. That kind of success might not have driven me out of the industry and into the welcoming arms of computers. I’m not a lot better typist, but the delete key does wonders for my self-confidence.

And let’s be honest about what we’re automating: It’s literally the most boring and least useful work we do. I wasn’t exactly high skilled labor, but I assure you there were more valuable things for me to do than try to set a nail while on my knees, bent over to the base molding. As a SysAdmin, yeah, my bosses thought my job was typing the same command 1000 times a day, but we can look back now and see that definition actually got in the way of the work, rather than being it. We had much more important stuff to do. Or at least, I did. Not sure about the bosses.

If we were all willing to drive cars from the 70s, while living in houses from the 70s (oh god the colors), using computers from the 70s (all three of them), watching channels from the 70s (all three of them), then yeah, maybe we could also work 5 hours a week.

But we all know what we’d do with the spare time: We’d make more work.

You’d do something silly, like write a book. And obviously, most of those books would be junk, but enough would be good that it would become someone’s job. And maybe the other writers didn’t actually have a knack for writing, but found they could be great at helping other writers. Oops, now you’re an editor, or a publisher. And now you’re working more than five hours a week again!

Stupid entrepreneurship.

Or maybe you don’t want to be a writer, you just hate Avocado Green enough to do something different with your kitchen. Oops, now your friends want the same thing, and you’re an interior designer.

Or maybe you realize that the gas-guzzling death-traps we all drive in are insane, and you figure out some way to start bringing those sweet little rides over from Japan. You know, just to fill the time. Because you can only consume so much of your day watching three channels (and remember, no ESPN in this scenario).

You’re not the only one that’s bored. Everyone else has more time, and a need to fill it. They can spend the effort to become experts in watches, furniture, bicycles, hiking, boating, economics, philosophy, or any number of other areas. And we all know the main outcomes of expertise, beyond insufferable newsletters: Demand for more specialized stuff, or enough disgust at the lack of it that you’ll just do it your darn self.

Some number of us won’t do the work, but we still want more, because why else would I have read three books on horology? It only takes a few people to step in to fill that demand, and suddenly your time is taken up.

And of course, if you want to buy that fancy Patek Philippe to pass on to your kid, you probably need to work a few more hours than the minimum.

So now you know why we don’t work five hours a week, and hopefully you have a little more confidence that automation will generally be good, not bad. It’ll create more entrepreneurs, raise wages, increase quality, and reduce cost. Not every time, but most times.

It doesn’t explain why those prognosticators in the 70s were so silly, but, well, I bet it wasn’t the worst decision they made that decade.

  1. 16oz Estwing finish head with a curved claw, natch.

Decentralize Your Data

What does owning your data even mean, and can the Blockchain help?

Photo courtesy of Dennis Kummer

Congress recently required Mark Zuckerberg to defend his lifelong practice of mistreating your private information. Movements to give you control of this critical data took the opportunity to claim they can prevent future such breaches. Blockchain is the new solution in search of a problem, and personal data is in the crosshairs.

But can the blockchain actually help secure your personal data? What would that take? And seriously, what do people mean when they say we should own our own data?

It sounds nice. Too bad it won’t help. The problem is not “ownership” (whatever that even means in a world of infinite digital copies). It’s centralization. Having one person’s data is a small threat, only to that individual. Having everyone’s data is a national crisis.

By now we’re familiar with the huge amounts that Facebook, Google, Amazon, and apparently everyone except Apple have on us. But how did they get it? Mostly, we gave it to them, through using their products. What we didn’t give to them, we gave to someone else who then passed it on.

There have been massive breaches at Equifax, Facebook1, and many others. Even the general public is becoming aware of the real causes. Some of the the largest companies in the world exist purely to collect your information and sell access to you based on it. They might not sell your data, but they definitely sell your attention using it.

These are the problems you know about. Don’t worry; it gets worse from here. If you think your birthdate and pictures of your kids are personal, what about your DNA?

Anne Wojcicki is married to a Google founder, and she liked their data accumulation so much she started her own company to build a huge pile of even more personal data. 23andMe does not scrape the internet — or your cheeks — to get your DNA; no, people pay for the privilege of giving it to them. Yes, they offer a service in return, but do they clean house after? Hah! No. They keep it. (Hopefully somewhat more safely than Equifax does.)

What’s so wrong about there being a database of DNA from a big chunk of the population? Let’s ask the police.

You might not be afraid of the police. You should consider yourself lucky. I know anyone of color in the US is and should be. I know I am; I grew up on a commune, and policed raided us using helicopters and assault rifles in hopes of busting us for cannabis. I don’t mean to imply that hippies have been as systematically oppressed as African Americans (and certainly not in the south); just that I grew up with my own justified skepticism of exactly what that force was here for.

Even if you don’t fear the police, you should fear the consequences of DNA testing. The science behind most parts of DNA are absolutely rock solid2. The police work is another matter. Beyond outright fraud used to wrongly convict people, the messy world of testing DNA at crime scenes just makes it hard to get correct results. Juries inappropriately treat a complicated test as foolproof. It could be compromised anywhere from the crime scene to police handling to the lab itself. The failure rate even without fraud is high enough that I would not want to trust my life to it.

Not to imply that DNA testing is worthless; quite the contrary. It has been used to exonerate many people who were incorrectly imprisoned and put on death row. It’s not that it always fails, just that you don’t want to finding yourself gambling on it against life in prison.

But remember: This is just for cases where someone has a single person’s DNA. Like having just your fingerprint. What happens when someone like 23andMe has a whole database of it?

“If you didn’t do anything wrong, then you have nothing to fear.” Pfft. Yes, it starts with requests for the DNA of individual suspects, but it escalates to doing a database-wide search for DNA that matches. And by ‘matches’, we don’t mean, “is 100% guaranteed”, we mean, “eh, it’s pretty close”. A DNA “match” directed the police to someone they thought was a relative of a suspect, who was then brought in for questioning. So I guess as long as you’ve never done anything wrong, and aren’t related to anyone ever doing anything wrong, you’re fine. Right?

I feel so much better.

I had investors literally laugh at the idea that collecting this data introduced security concerns. They grew up at Google, so it’s not surprising they could not see centralization as a problem. Just like Equifax started out wanting to make it easier to get loans, and now they’ve got so much power you can’t get one without them.

There is a world of difference between giving someone your data, and allowing someone to include your data in a massive pile of it. Any discussion of the risks of data needs to acknowledge that.

Now we see our discussions of owning your own data don’t quite have it right. What we actually want is decentralization of data. We don’t want a single company to have access to this much information about huge groups of people.

And now you see the problem.

New technology can’t break Facebook’s business model. It can’t prevent Google from scraping every web site on the internet and identifying you by connecting everything. Whether you give it to them or not, they’ll know what you look like, where you live, and who you hang out with.

Most importantly, it can’t prevent people from sharing all that data with these services. After all, they’re getting something valuable in return, like connecting with friends and family. Or figuring out their family tree.

The problem is not the centralization. It’s the effectiveness of a business model built on centralization.

So anyone who comes to you and says “The blockchain will allow you to own your own data!”, ask them in return, “How will you make it such a joy to use that Facebook will go bankrupt?” And please, record the conversation, because I want to see them stammer.

This is fundamentally a product and design problem, but the technofuturists are treating it like a technology problem. “Oh, if only those college students had access to better cryptographic tools they never would have shared that data with Facebook!” 🤯 No. People will stop using Facebook, and 23andMe, and Google, when there are better solutions. And unfortunately, they need to work ten times better, not just a little bit.

So talk to me about the blockchain. I really do want to hear how you’ll use it to help people own their own data, and remove the incentive to centralize all of this data.

But talk to me of products. Of user benefits. Of business models built around all of this.

Because people have to want what you’re selling, and the only way to get that is to build something they want to use. Only then will they be able to own their own data.

This is the third article in a series of indefinite length on The Blockchain Without Blockchain

  1. Although technically not a breach, since their usage rules weren’t broken — that’s how little they respect your privacy.
  2. Although we’ve still got a lot to learn about the epigenome, so don’t think we’re done here.

Trusting more with the blockchain

Society is built on trust, and improves or weakens with it.

Photo by Nathaniel Tetteh

I know I have trust issues. I don’t need the blockchain crowd telling me.

Trusting is scary. We’ve all been burned at some point. But we can also look back and see trusting someone helped us develop, personally and professionally. None of us could be who we are if we had not learned this critical skill. Knowing where and how to trust is critical to growth, to life. It’s not even just humans — we can see this in our pets, our livestock.

A cynic might say that trust limits us. That if we only had less, we could do and be more. I’m not exactly known for looking on the bright side, but even I know this is wrong. Trust is the infrastructure for our experiences. Removing it flattens everything, not just limiting what you can do but limiting why you would do it.

Our problem is too little trust, not too much.

We know the stereotype of someone who does not trust. Someone outside of society. We know a person who cannot trust is broken in some way, missing something critical, in need of healing. Many of us also know the allure of not needing to trust, or be trusted. “Ah, to be independent, to owe nothing to anyone…”

This is the dream of remembered childhood. It was always a lie. We were failing to notice the work being done in our name, for us. It was a joyous lie, made more pleasant with the golden tinge of nostalgia. Grown, we miss the lie, we reach for it.

But deep down, we know: More than anything else, life is about trust.

Great companies have been built on this truth. eBay could only exist by creating trust between unknown parties.

Some look at this and see failure. “If only eBay had not needed trust…”

One of the Blockchain’s great claims is enabling commerce between people who don’t trust each other. Never mind that of course you still have to trust something — the code, the packaging of what you’re buying, the exchange, etc. You might scoff and say these are a given, but none of those things can be trusted in the current world of the blockchain. Never mind that commerce has always been done between people with little or no trust. That’s not what matters.

It is philosophical, psychological: Given the recognition that life is enriched by trust, and more riches require more trust, what do you do? Find a way to add trust to your life, or look for a way to get riches without it?

I can’t say the blockchain people are wrong. Maybe they really do need some kind of trustless commerce. I don’t know them. Well, other than the drug dealers. I know why they want this.

But in my life, for my problems? More trust is the answer, not less.

Ironically, the blockchain can actually help with that. Without changing a thing. Its boosters are right about its utility, they’re just wrong about why it works.

I don’t like to trust people with my data. People talk about wanting to own their data, being able to share bits with Facebook but not the whole thing. It’s a nice, if naive1, idea, but that’s not what I mean.

I don’t trust you to touch it. You’ll muck it up.

Heck, I don’t even trust myself. Actually, I was never given that choice. My apps don’t trust me with my own data. They keep it hidden away somewhere, behind an API, in proprietary formats.

Their distrust is reasonable. I don’t know how the app works. The data model is hidden, the storage internal. Most importantly, they can’t tell if I mess with it, and they can’t fix what I break.

Things were in some ways better in the age of documents, but now our data is all hidden. We ask them to give us access, and they sometimes comply with simplistic APIs. But they do not trust us.

What if they did? What if I were allowed access to my own data? What if I could share it with you, my close friend, because I trust you with it?

I mean, not entirely trust. I’m not stupid. We’re not that close.

With the right tools, I could see what you did, understand it, ensure it all makes sense. You could change it, query it, hand it back to me, and I could validate the whole thing. Get the best out of your work, but keep safety lines in place.

Again, a cynic would say call this an example of eliminating trust. But is it?

Is the key to this new interaction really that I don’t trust you?

No. I don’t want just anyone to have my data. It’s for you. My close friend. Who I already trust. Mostly.

This change does not entice me to share with psychopaths, strangers, or, god forbid, the people I went to high school with. It provides just enough of a bridge that I’m willing to give you, my good friend, who I just met on the internet, rights that I’d otherwise hold back.

Of course I know this is not what blockchain people mean when they talk about trust. Meh. I’m not interested in making capitalism even less moral, less human. I don’t even want to hang out with the people who do. But I am interested in making data more useful. And I’m especially interested in connecting with other people.

And this certainly does that.

Now my applications can expose their insides. They can be slugs instead of snails.2 I can use the apps I love, but my tools can fill their gaps. I can script my way around their missing APIs and limited reporting. Heck, I don’t need their interfaces at all. Just because they’re provided doesn’t make them good, and I can get there faster using the tools I already know.

I can pick the best app, without committing to a long-term relationship. I can give you my data, take advantage of what you offer, and if I want to change later, I don’t lose everything.

There’s a great example out there: Github. If they went away tomorrow, I would lose almost no data I care about. They host, but do not control, my most important data: my source code. I get some utility out of they’re hosting it, but they don’t get special rights.3

Github actually created a new kind of trust relationship: Because its users can trust the data store, they began to trust strangers to contribute code. If I were using Subversion, I would have to give you all or no access; in Git, I can give you qualified access. Github calls these ‘pull requests’. “Yes, you can contribute code, but I get to read it first.” That enables a flowering of trust, potentially leading to a deep relationship. That path to complete trust is much narrower, much harder without this infrastructure of gradual trust.

You can choose whether to see this as more or less trust. You can’t argue with the new bonds created, the new groups formed, all through the help of tooling. Almost like how commerce starts with low-trust exchanges of money and can lead to deep and meaningful relationships.

What would a world look like if all of my applications had just as much, or as little, ownership of my data as Github does?

Of course, the apps won’t like it much. Their trusting me with my data also gives me power over it, where today I have none.

With enough usage, our expectations start to change. Given two otherwise equivalent accounting apps, wouldn’t you pick the one that trusted you? That gave you equal access to your data, simplifying automation and reporting?

I would.

This is the world I want.

This is the second article in a series in indefinite length on The Blockchain without Blockchain.

  1. Facebook does not have all of your data because you provided a massive dump at once; they’ve painstakingly collected it over the years, bit by bit. You can be sure they’ll store every little piece you selectively reveal to them.
  2. Honestly I don’t know if slugs are more useful, but they’re certainly more vulnerable. And I could not think of a better analogy.
  3. There is some data that only they have. This is a limitation in git, more than the plan of GitHub. E.g., my follower list is not in my repo, which is probably good, but it’s not anywhere else I can access, which is probably bad.

Blockchain without Blockchain

There is powerful technology hidden in the blockchain. Just strip out the parts everyone cares about

Image courtesy of MemeGenerator

People are very excited about the blockchain, convinced it’s going going to replace cash, or gold, or credit cards, or even the internet. I’ve even read claims that it’s the most important invention since little things like money and democracy. Starving through a failed attempt at utopia has taught me to align with the skeptics who point to the lack of proven use cases (other than fraud and crime), and with the pranksters who make fun of the claims of salvation.

That doesn’t mean I think it’s worthless.

The economic use cases (cash, gold standard, “sound money”, etc.) rest somewhere between unproven and conspiracy theory. None get any love from me. Just because I don’t respect the Austrian school of economics doesn’t mean I don’t understand it. They’re so stuck on their commitment to not learning from history that they can’t let go of the switch away from the gold standard. I am also not silly enough to think that the anti-government libertarians are in it for anyone but themselves.

I’m only here for the tech, people.

The world of the blockchain is in a state of flux. This is normal for movements, and especially so for the definitions of words to be shifting quickly. I experienced this directly as Puppet was helping to create the DevOps space. Today, when we say ‘blockchain’ we mean a specific set of features provided by the code underlying Bitcoin; some might add the requirement for smart contracts, as added by Etherium.

This definition won’t last.

I think that when we look up in five years, the word will mean something quite different. As the conversation inevitably shifts from hype about its universal applicability to finding practical use cases, I am convinced the term ‘blockchain’ will invert from its maximalist form, including all possible features, to a much more minimal form, defining a base set of functionality that is suitable to solving far more problems. Different implementations will layer features onto that base set, but all of these derivatives will be called ‘blockchain’ (which will soon have purists crying we’re not using the “real” blockchain).

It’s that base set of functionality that I’m interested in. I’m hoping the world’s excitement over the complete package can turn into momentum around a more flexible solution. What can we take away, and what can we do once we strip things down?

Let’s start with the most obvious thing we don’t need: Lack of trust. Free markets are built on trust, so it’s silly to try to build one that doesn’t. I just don’t run into many problems that can only be solved if I never trust anyone. Even writing that out makes me cringe.

I’m interested in power tools, user productivity, and helping teams, which inherently means I’m interested in people who know and trust each other. That isn’t to say data validation and simple synchronization are never helpful, but trust between people is not a problem. So take out that whole part of the story.

Next up is consensus. Let’s be honest: You don’t really want the world to have access to most of your data. And most of you who do, well, it probably doesn’t want to see it. This solution for deciding whose copy of data is right? I don’t need it.

That isn’t to say there aren’t important consensus problems; it’s just that the blockchain’s version of them is not useful to me. I do actually struggle with managing conflicts in my own work. I’m writing this essay on one of the six devices I regularly work from, and I obviously want it to transparently synchronize between the rest. I can dream about never having conflicts, but in the real world I need a simple way to handle them when they happen. That’s my consensus algorithm. If I use the Blockchain’s, then one chunk of work wins and the rest of the work gets thrown away. That might work well in some environments, but is obviously a non-starter for my personal work. Instead, I need an algorithm that gives me control over managing my own work streams.

I expect a blockchain hodler would flip at calling this a “consensus algorithm”. It’s not. It’s more like a tool for managing merge conflicts. If you buy into the blockchain, you’ve added a ton of complexity that can and should be handled by a person.

It’s not a big leap from managing conflicts between one person’s devices to managing conflicts within a team. This is still an important problem, one wrestled with by anyone building online collaboration tools, but requires nothing like the level of infrastructure built into the blockchain. After all, I trust myself. I trust my team. I like to be able to verify, but I’m more worried about mistakes and data loss than I am about intentional subversion of the system. And actually, the current solution is horrible for teams. It’s one thing for someone else’s work to cause yours to sit in a queue for a bit, it’s another thing entirely to have your code thrown away because someone is in front of you.

Taking out the trust-less consensus allows us to remove the worst part of bitcoin: Proof of work. This is how parties in a blockchain transaction fight to have their work accepted and others’ rejected. If they win, they are rewarded with a token, which is how they’re “paid” for their work.

We don’t need any of that.

We’ve already established we don’t need an automated, trustless process for deciding who gets to update the database (and we’ve concluded we don’t want conflicting just thrown away). As a result, this whole mechanism can just be removed. Good thing, too, because bitcoin is in the process of consuming all of the world’s natural resources in the name of never trusting anyone. This also simplifies the problem of scaling these databases. Bitcoin is stupid slow because its proof of work system is stupid slow. Remove that, and any iPhone can trivially record new data as fast as you want.

Now that we don’t need proof of work, we also don’t need, surprise!, the currency itself. In a trustless system, tokens are used to compensate the networks recording the transactions. This is now so cheap we don’t need to reward people to do it.

There you go. Now you’ve got the blockchain, except without cryptocurrencies, trustless transactions, consensus algorithms, or proof of work.

The blockchain without blockchain.

It’s important to note: It’s not that I think none of these features are ever useful in any case. It’s that none of them are necessary to get the most important features out of the blockchain, and each of them should only ever be added to a system if they’re truly needed. In most cases, YAGNI.

What can we do now? Well, obviously, now that we’ve removed so much functionality we can do a lot more. General purpose languages are more powerful than specialized ones, and a more flexible database is a more powerful one. The current set of blockchain features can really only be used for a narrow set of use cases, and even those seem to be more theoretical than actual.

What’s left?

A database. That’s what the blockchain always was anyway. It’s built on Merkle trees, which means you can validate every change back to initialization if necessary. This makes it kind of trustless: I can validate your work, rather than just trusting your word of what you did. This ability actually encourages me to trust you more with my data, though.

Check back later for more about what I mean.

This article is the first in a series of indefinite length. I’ll explore the consequences of this stripping away. I will also address some of the potential I see for full current blockchain feature set. I look forward to approaching this topic from a different direction.

The Market Is Wrong About Your Problems

What Voltaire and the Flaw at the Heart of Economics Have to Teach Us About Software That Doesn’t Exist

Voltaire’s Candide juxtaposes an optimistic philosophy with unbelievable tragedy. He was angry at the 19th century philosophers who proclaimed that we lived in the best of all possible world while destruction and death unfolded around Europe on an epic scale.

We might hear the claim that we live in the best of all possible words and scoff. Of course, we’re too enlightened to be such naïve optimists. But are we? Isn’t the belief tempting? Or even, doesn’t the behavior of those around you make more sense if you realize they believe this, at least a little bit?

Economists are theoretically rational, analytical, big picture thinkers, but at the root of modern economics is a belief shockingly close to Candide’s parody of optimism. They have what they call “The Efficient Market Hypothesis” (EMH), which roughly states that all assets are valued fairly. This is built off the idea that asset values in an open market are fair because they include all available information, and all the actors in that market are behaving rationally in regard to both the asset and the available information.

This theory tends not to trigger the cynicism that Voltaire does. Intuitively, it sounds not just right, but defined as so. Isn’t an open market essentially a mechanism for finding the fair value of an asset? It’s not so simple. And when it goes wrong, it does so spectacularly.

Modern economists cannot be as destructive as the great thinkers of the 18th century, whose big ideas justified eugenics and many other horrors. Just because they cannot as easily be used to justify mass murder does not mean they should not be accountable for the downsides of their obviously incorrect theory.

“No”, I hear you say, “the EMH is not wrong; it’s correct by definition.”

Economists have convinced us of what Voltaire was protecting us from: We live in the best of all possible markets, where all information is public and all assets are fairly valued. If the market does not value something, that it must actually be worthless.

But of course, if that were true Warren Buffet would not have become a billionaire buying stocks that were worth more than the market was paying, the finance industry could not have been built on advising clients about public stocks, and you’d have no need for lemon laws or other regulations that fight information discrepancies. Nor would Kahneman and Tversky have won the Nobel Prize for demonstrating that actors in an economic system behave anything but rationally, puncturing the EMH for good. Thankfully, this has forced the field to begin to grapple with its flawed underpinnings, but many modern beliefs are implicitly built around these bankrupt theories.

You might be patting yourself on the back right now for not being silly enough to draw Voltaire’s ire, but it’s baked into the value system of the world around you, especially if you live in the US.

  • The market moves from irrationally ignoring new technologies like the blockchain to irrationally dumping money on them, without any fundamental change to justify the shift
  • Investments are made based on proximity and serendipity rather than rationality and opportunity size
  • We tend to claim that the rich earned their status through hard work, rather than recognizing the role of privilege, inheritance, and luck in their status

Of course, not everyone in the market operates with such optimism, but each of us is biased in this direction. It affects our thinking whether we want it to or not.

“Ok”, you say, “even if I accept some people make optimistic investment decisions, what does that have to do with software?”

Great question. If we live in the best of all possible markets, where all information is public and all assets are fairly valued, then we can trust the market’s assessment of what software should and should not exist. Lack of software to solve a problem is a sign that it’s not worth solving.

If, on the other hand, our world could be better, or if our market is imperfect at valuing assets, then we can’t trust intuitive conclusions about where value resides. This is most true when it comes to valuing unsolved problems. It might be that a given problem has no solutions because it is not worth solving, but mundane reasons are more likely to be at fault.

Most great companies exist because they provided something the market did not know it wanted. Their founders encountered a flaw, and managed to build something great in the opportunity created by it. Henry Ford claimed if he’d have given people what they wanted it would have been a faster horse. The market knew how to value them, but not cars. Before Apple, the market did not value personal computers. Before Google, the market valued directories but not search engines. Before the iPhone, the market valued expensive phones for professional use but not personal.

These value statements were market failures, and their resolution generated billions of dollars for the companies resolving them. Now, of course, the market sees great value in what these founders have created, but not because the market is so smart; it’s because it can no longer fool itself.

It’s easy to grow despondent in the face of such obvious market failures. If the wisdom of the crowds, the great invisible hand of the market, can be so wrong, what hope does a lonely entrepreneur have? I take a different way.

I luxuriate in these misses.

They surround us. We bathe in them. Yes, many great companies have grown into critical market gaps, but even with all these successes, there are untold problems whose solution should be valued but is not.

Only once you reject the market’s flawed opinions about what matters, you begin to see nearly limitless opportunity. There are so many more unmet needs than there are perfect solutions. These are your opportunities.

Of course, just because the market dismisses a space doesn’t mean there’s a great opportunity there. It’s your job to know the problem, your customer, your user, your buyer well enough to draw your own conclusions, to develop enough certainty that you don’t need someone else to tell you what to believe.

Because that’s the real point: Trust yourself, not a bunch of paternalistic optimists.

Why We Hate Working for Big Companies

Modern capitalism raises the flag of the free market while pitting centrally planned organizations against each other

It’s quite a journey from being born on a commune to raising more than $87m in funding at a software company. This journey forced me to wrestle with existential questions about my true beliefs, and how they intersected my life as an entrepreneur. One’s work is rarely a pure reflection of ideology, but companies need a clear and authentic strategy, which requires a tight alignment between company operations and the founder’s philosophy. I have discovered more about those differences between what I believe and the best ways to grow a corporation while studying economics — that is, how money is made and exchanged — than any other area.

A worldwide conflict between communism and capitalism defined the latter half of the twentieth century. The United States’ ideological battle was the central drama of my childhood, and it was with a combination of glee, pride, and “told you so!” that my fellow Americans watched the wall fall in Berlin, and the USSR dissolve shortly thereafter. I expect few would deny that the US is the standard bearer for capitalism.

Yet, there’s a flaw at the heart of this claim. While the United States operates as a free market economy, the key agent within modern capitalism — the corporation — works more like an authoritarian state. Given how much of our world is built around corporations, this truth and its impacts are critical.

I grew up apart from America’s passion for capitalism. In the era of Reagan, I was living on a commune. My parents did not earn money for their labor, and we didn’t have personal property. My family left the Farm when I was 8, and as I matured, my ideological roots were in conflict with the US’s nonstop pro-capitalism message. As I joined the workforce and eventually started my own company, I found myself attached to neither the communal roots of my childhood nor the Wolf of Wall Street world I moved into. I grew slowly in convictions, as I encountered problems in the course of scaling a company.

The first real conflict came when it was time to hire managers. I founded a company primarily because I did not thrive as someone else’s employee, so what led me to think others would? More importantly, anyone who has ever operated at the front line is aware of the severe costs imposed by the separation between the people who do the work and the people who make the decisions in hierarchies. Hiring managers was just going to make the company do worse, not better, right? Right?

I expect three of you are gleefully shouting, “Yay, holacracy!” right now, while the rest are confused and either offended or think I’m an idiot. I did consider a manager-less world, but a little research provided only examples of disaster, because the only available options just replace an explicit power structure with an implicit one. In other words, it’s still hierarchical with the founder on top, but now decision making is opaque and the system is easy to exploit because of the lack of controls (which looks surprisingly like the cult/commune I grew up in).

Those who are confused or offended by the idea that managers make performance worse would be informed by a deep dip in economics. One of the core principles of the free market is that central planning committees can never be as efficient or as effective as the people doing the work. By definition a free market economy lacks a decision-making hierarchy; the ‘free’ means every agent (individual or corporation) can decide for themselves, without needing permission from a manager above.

While there are many aspects of modern American capitalism I reject, this one I wholeheartedly support1. The downsides of a strong central executive were taught to me early.

Like many other communes, the one I grew up on routinely failed to feed its people — my parents speak with horror of the ‘wheat berry winter’, when we lived on little else. While his people were short on food, the founder of the Farm was off touring Europe as the 3rd drummer in a band, “bringing our message to the world”.

Thankfully none of us starved to death, but the failing was similar to what most communist countries experienced: The central organization could not feed everyone. For years, I assumed this was just incompetence, whether at the scale of the Farm or China. The truth was far more structural. Millions starved during the Great Leap Forward because the central organization was trying something impossible: Managing the productive output of an entire country. The Planet Money podcast tells a great story of how this central planning was walked back in China, but the general point here is that these communist countries did not just nationalize the means of production, they tried to centrally control all of it from within a small group.2

When people talk about communist countries not being a free market, this is what they mean: They tell the farms what crops to produce and in what quantity, rather than letting them decide for themselves. China even went so far as to dictate what hours a farmer should start and stop working, and then directed managers to ring a bell for transition times to control every little group of farmers. Anyone who’s ever had to punch a clock into a rigid, dysfunctional hierarchy is likely getting painful flashbacks about now.

It should be immediately obvious why this fails miserably: The distance between the central planning committee and the farmer is so great that good decisions are nearly impossible. It’s nearly impossible for critical feedback to make it from the edge, where the farmers are working, to the central planning committee in time to affect decisions, and then for those decisions to make it back to the edge in time to be useful. The podcast linked above also points out how unmotivated the farmers were under this regime, cutting productivity even further. Those who have studied lean manufacturing, agile development, and DevOps are likely seeing parallels here.

The result was catastrophe. When a corporation is painfully inefficient it loses money and might have to do layoffs, but when a country fails at growing food, its people starve to death. I don’t mean to imply that central planning was the only cause of famine under communist rule — there were political operations that led to mass starvation, just like in the West — but learning more about these helped crystallize what I do truly prefer about capitalist models. It also converted the phrase ‘the free market’ from a catchy slogan into something meaningful to me.3

The most important feature of free market economies is that each person within them is able to make independent decisions in their own best interests4. If you’re a farmer, you can decide what to grow, how much to grow, and when to work to develop your crop. Heck, you can even choose not to be a farmer any more. Success is merely dependent on your finding a buyer for your work at a price you can tolerate. Any given year might not be perfect, but your decision making gets better over time as you learn to respond to customer demand.

This pattern is easy to understand in any system where the people doing the work make the decisions. If you’re a jeweler, you can decide what to make, how much to sell it for, and what to spend your time on. Same if you run a small restaurant, lead local tours, or are a one-person shop doing house remodeling. It’s a free market, where you can charge what the market will bear, and you can quickly and efficiently respond to its whims, ensuring that you are getting the best use of your time.

This was a powerful organizing principle for a long time. The history of human commerce developed largely this way: One person, or as many people as could fit in one shop, would turn labor into a product, then find a buyer for it. Most large-scale efforts were organized by the state of the time: Monarchs and the landed gentry, who were the only ones capable of marshaling enough resources to build palaces, roads, and other large construction projects.

This began to change in the 17th century when corporations like the Dutch East India Company were able to deliver massive windfalls to investors by pooling money and using it to extract resources from colonies. There was a step change in the 19th century, as corporations went from generating wealth to building and owning infrastructure. It’s one thing to outfit a single ship for a year-long voyage, yet another to maintain railroad schedules across the United Kingdom, or run a telegraph network around the whole US. These aren’t just short-term money-making exercises, they’re long-term commitments with big capital outlays and large returns over years and years.

We still live in a free market economy, but it’s not one Adam Smith would recognize. Instead of individual or small operators, ours is composed almost entirely of corporations. Really big corporations. And these companies, they use the same kind of central planning that we so despise in communist systems. I know. I’ve done it.

By the time my company got near 500 people, we had a multi-week planning process, where the leadership (i.e., me and my lieutenants) set out top-level goals, built a top-down plan to accomplish them, then drew information from the front line to see where it needed change. We called this a bottom-up plan, but it was only bottom-up from the perspective of numbers — how much money we’d have, what our costs were, etc. — rather than from the bottom of the organization. We could see no way to have a system where the people doing the work built a plan for the organization. Even thinking about it now, my reaction is, “How would they know what my goals are?”

That’s the kind of question you can only ask in an authoritarian state, not in a free market economy. My goals became my company’s goals, and the only real way to ensure people worked toward them was providing a plan. You might argue that a corporation should focus on shareholder value, but that doesn’t help make decisions about what the company should actually do.

Great leaders find a way to listen to everyone in the company, but in the end, leadership is about making decisions. That’s essentially the definition of the word. And we all know leaders who did not bother to listen, or just did not need to in order to be great; today’s most vaunted tech leader, Steve Jobs, was famously disrespectful of the opinions of others, yet made a lot of world-changing decisions (not all for the better).

This is exactly why working in a big corporation is so stifling. If you’re in a small company, the executives are close enough to the front line that it’s more like working in a tribe, but in a big company, the leadership is so removed from whose who do the work that executive teams operate like the politburo we so decry in communist countries. Certainly the bureaucracies are no more enjoyable or forgiving.

I find it both ironic and painful that my inability to work for someone else resulted in my creating a company that involved a lot of smart, capable people working for someone else.

I wish I had a solution. If this were an easy problem, its solution would already be pervasive, because the benefits are massive. Just in terms of efficiency, we’ve seen how much better the free market is than planned economies, but it also has a hugely positive impact on quality of life. People are happier when they’re in control.

I know the solution is not more freelancing and contract work, which America’s corporations are addicted to. That’s the worst of both worlds: The exploitative nature of capitalism with the inefficient bureaucracies of communism. Transactions on the free market work because they’re good for both sides, but most people only accept part-time contract relationships today when they have no other real choices.

Holacracy certainly isn’t the answer. It’s fundamentally flawed because of its implicit power structure — Tony Hsieh still runs Zappos, even if he does not use a central planning committee to do it — but the biggest problem is it makes no mention of economics. Without a clear system for scoring the transactions (i.e., money) it’s impossible to build a free market.

This problem of how to handle economics within a non-hierarchical company might lead some to think of using blockchain tokens as an internal currency. This is impossible today, beyond the fact that the world of blockchain is mostly about fraud and black market sales. The biggest problem is that we have no idea how to value most of the work people do. I mean, we might know that what a developer should get paid for a year’s work, but how much is that work worth? The majority of the work done in modern corporations is incredibly hard to value, which is partially why companies are so inefficient and make so many bad decisions.

That brings up an even bigger problem — companies today hire workers to make money from their labor. In other words, they generate profit because they pay their employees less than they’re worth. If everyone could trade their labor for exactly the amount of money it was worth, the corporations that employ them would have a much harder time making money. Instead, in modern corporations the shareholders and the executive team — again, the central planning committee we so despise — make the majority of the money, while the front line does all the work and makes very little. This is true even at the big tech firms; software developers might be well paid relative to hotel workers, but they’re paid a pittance compared to the founders and executives. This might speak to why we have no solution yet — free market corporations would tend to reduce concentrations of wealth, which would be terribly disruptive to the current system.

Like I said, I don’t have a solution. But at least now I know what makes the current system so painful, and it gives me some hope that we actually can come up with a better answer. I know I’ll be working harder in the future to manage the downsides of what we have today.

  1. Although I might stress the “well regulated” part more than most modern economists.
  2. Of course, capitalism is just as capable of killing its citizens, whether through starvation or lack of health care.
  3. Note that I’m not taking the capitalist side of the cold war here; while Americans were decrying the oppression of the Soviets, we were actively clawing back progress on civil rights and knocking over democratically elected governments. This article is about principles, which political regimes rarely show a great track record in following.
  4. But not so independent that you should be as pathological as Ayn Rand.