This is my reply to Jim Antonopoulos' article Is The Creative Director Dead?, which you should probably read first. I subscribe to Jim's newsletter because he always has insightful thinking on branding, strategy and design.
I saw work sold into my employer where the project-winning hierarchy of people entered from their conquest to great applause and then promptly went to lunch, with the so-called brief delivered in a paper folder by a 21 year-old account person—who was not at the conquest in question—translating it to me, the designer, the following Monday. And I wasn't able to speak to the Creative Director (CD) about his brief until Friday. What you describe in your article, Jim, is the Principal-Agent problem and I have been the very definition of the Agent.
That is, I have been a "pyramid builder" not a "Pharaoh", to match your terms. Much of the incentive to work at studios was—"was" because I don't have that incentive anymore—based on social status because most young pyramid building designers are a bit "edgy" (this is the lingo of the client-side Suits) and see the world differently, and thus have the skills to render "newness". And these studios not only sought talent based on the premise of status, they traded on it as a business.
The design business trades on status
A business' status is always important. But within a domain that is difficult to assess, such as design, it is more important. A graphic designer's value is in creating emotion to communicate with an audience. But often the people who pay for their work don't care what it looks like because they buy the outcome. And that outcome—emotional resonance—is a difficult thing to measure. So buyers typically have no real way of knowing that what they buy works, apart from their own intuitions of it. So they defer, among other things, to a studio's status.
In this way, graphic design is akin to the fashion industry. We have the same type of Pharaohs: original geniuses whose taste is ascendant and ahead of the What's Happening Now curve. If you're winning that game, you're brilliant. But from the outside, it's pretty easy to be a little cynical and describe this process as selling microtrends to suckers.
Information Theory and the new Market for Lemons
While this "market for lemons" scenario worked for most of last century, today we live in an hyper-information society. Our society is now exploding with information from the first 25 years of the public internet. Because the internet is software, so software workflow terms like "agile" are now well known and the lingo is understood enough to be implemented across non-software domains such they really do translate into business terms like "delivery" (examples are countless and not required as part of this short note).
Technology has eaten out more than a little of the flesh of the design industry. Everyone is now a media producer. So everyone can be a CD too, as you said. Cheap tools and software that makes it easy to mash together a graphic in 5 minutes without resorting to employing "talent". But by constantly searching for The New Thing, designers play a hyper-game of diminishing returns.
As such, graphic designers have a crisis of confidence. As this Walker Art Center article (quoted below) shows us, graphic designers have failed to realise that information systems do not need them anymore.
blogs … their rapid publishing pace feels disposable
It is much harder to get clicks with a long-form article than it is with a handful of images of sexy typography, and many sites create and contribute to this negative feedback loop.
By chasing the same old Fashion Aesthetics game without fully realising that our information society is utterly different to any other society in human history, CDs are remnants of a past era. They're bottlenecks, as Ben Terrett says. They mistake the map for the territory because they don't know how the territory works. More so, they're afraid to know about this new territory so they ignore it and double-down on attempting to become a Creative Original Genius.
Start with problems
As the Quantum physicist David Deutsch said, all problems are soluble according to how interesting they are. The solution to CDs finding a part to play in the creation of emotion in their client's audience is first of all understanding that the ground has shifted: the problem they're solving for is now a different problem. And from there, begin reframing that problem, as you are doing, Jim. Or find more interesting ones. I like this quote from Brian Collins:
We see design as a problem-seeking process. Seeking, asking increasingly bigger questions, allows you to reframe a problem.
By asking "what if", we start by defining the problem first, well before we have fun figuring out what it looks like. That starts by being interested about how the world works right now and making some guesses about why it works like this. That is, to start seeing again. To anticipate, to seek, to conjecture.
It is the purpose that makes strong the vow. Shakespeare
This cannot be done by reference solely to what a thing should look like. Instead, we should first attempt to explain what is we're attempting to do, even if it's to ourselves. At this point, we're able to get to the guts of what the word "design" actually means: to plan, to mark out, to propose, to intend.