Developer Productivity in 2024: New Metrics, More GenAI
There’s no doubt that 2023 saw increased obsession with developer productivity. That’s because tech layoffs came — and, unfortunately, keep coming — at a time of ever-increasing tooling complexity in the cloud.
Teams need to do more with less. And the resulting workload and cognitive load is nearing insurmountable. Something had to shift.
This has given rise to the sociotechnical discipline of platform engineering, with developer enablement teams dedicated to improving the work life of their internal developer customers — because, no matter what the macroeconomic climate, a lot of data show that happy workers are more productive ones.
Developer productivity efforts look to increase the speed of software development teams in delivering business value via high-quality code. This effort to remove barriers to software delivery will continue to scale in 2024.
It’s time to reflect on several developer productivity initiatives over the past year, to a backdrop of the freshest research, in an effort to predict the state of developer productivity in 2024.
The Rise of Platform Engineering
Platform engineering was a buzzing topic of 2023 — in the software development world and on The New Stack. We even wrote a book about it.
Like its name suggests, platform engineering is the scaffolding —usually in the form of an internal developer platform, plus team — that offers a strong enough base that the majority of development teams within an organization can reliably and securely build on top of it.
A dev team can choose to climb down a level to take a different route, but that means they are wandering off the golden path to productivity laid out by the platform engineering team — and thus won’t be guaranteed the same level of support and uptime.
Unlike the previous implementations of platforms, which were very top down and mandatory, platform teams treat their internal developer colleagues like customers, adopting a Platform as a Product mindset. It’s much more about building something these customers want to use by visibly enabling teams to be more productive.
Earlier this year, Abby Bangser, principal engineer at Syntasso, defined platform engineering as the team and technology that aims to abstract out the “non-differentiating but not unimportant” work in the software development life cycle (SDLC). The DX developer experience platform recently published a list of common types of developer productivity teams, which echoed that sentiment:
- Developer tooling: build, test, deploy.
- Enablement and internal support: codifying workflows and offering support.
- Frontend platform: UI components, web frameworks, search.
- Backend platform: authentication, web gateway, APIs.
- Infrastructure: Terraform, logging, Kubernetes, observability, cloud services.
- Reliability: incident management and site reliability engineering.
- Security: security standards and risk management.
- Data: data engineering, warehousing, access.
In the age of DevOps, the SDLC encompasses a lot of essential work that can distract from a development team’s main purpose — driving differential value to the end user. The platform team — which brings together some or all of these developer productivity functions — can standardize and automate a lot of this important work, so devs can feel productive and impactful faster.
But, be forewarned: a platform team is never short on feature requests, leaving it to prioritize for a combination of improvements that help reduce friction and increase flow for most of its in-house developers. That makes getting started particularly challenging.
Team Topologies — a popular approach by Manuel Pais and Matthew Skelton for modern, sustainable team structure — advocates for kicking off with the thinnest viable platform, something that can offer an early win. That, more often than not, starts with documentation or discoverability — elucidating what, how and/or who does what — and grows from there.
And this isn’t just a trend either. Platform engineering is increasingly invested in as the go-to strategy to drive developer productivity.
Port’s just-published ”2024 State of Internal Developer Portals” surveyed 100 engineering leaders at organizations with 150 or more developers about their platform and portal programs (an internal developer portal is the way in which a developer accesses an internal dev platform).
The study found that 85% of companies have either started implementing IDPs or are planning to, while 99% of respondents have at least begun their platform engineering journey.
What’s perhaps the most notable survey result is that 53% of respondents said they began that journey just this year, meaning that 2024 will likely bear the fruit of these platforms maturing.
Supporting Developer Productivity
One question that The New Stack has examined this year is the size of a developer productivity or platform engineering team. After all, at a startup, every dev may be contributing to the platform, just not as their full-time job. They’re embracing the idea of: See a problem, fix it, at scale.
In this author’s interviews across the year, most enablement teams made up less than 10% of the engineering org, with the exception of Netflix developer productivity, which has 15% enabling roles.
The newest report from DX is on “Sizing Developer Productivity Teams.” What they found, across 40 teams interviewed, was that companies with under 1,000 developers allocated about 19% of that headcount to centralized developer productivity teams.
At companies with more than 10,000 engineers, about 11% work on cross-organizational engineering enablement.
There’s no doubt that, just like the overall engineering it supports, developer productivity is a costly endeavor. But once it’s up and running it should be saving overall time and money across the organization. Logically, the work of a successful platform team should scale exponentially past its size.
The coming year will likely give more insight into whether the size of teams grows along with platform popularity or if platform maturity enables smaller teams to support larger groups of engineers.
The Metrics Debate Continues
“What makes a developer or team productive changes from team to team, from quarter to quarter, even from individual to individual,” Fernando Villalba, a DevRel at OpsLevel, recently wrote on LinkedIn. “Trying to impose accurate metrics is futile.”
Still, you can’t improve what you can’t measure. In a year obsessed with productivity gains, measuring developer productivity has been on everybody’s mind. In fact, a lot of good frameworks came out to do just that.
And then McKinsey claimed “Yes, you can measure software developer productivity” to an overwhelming response of, well, maybe one of the world’s largest consultancies can’t do that.
Yes, the McKinsey framework builds on the industry-standard DORA and the SPACE frameworks, but then it aims to create its own framework that concentrates on a narrow definition of developer productivity, where the most value developers deliver is building, coding and testing.
This is part of why, famously, developers distrust productivity metrics.
To compound these misgivings, Kent Beck, founder of Extreme Programming, told The New Stack that few organizations are asking and answering the key questions around the motivations for these metrics:
- Who is asking?
- What is their power relationship with those being measured?
- What is the purpose of asking? What decisions will change based on the data?
- What counter-measures will be deployed by those being measured?
- How will those countermeasures affect the goals of managers? Investors? Customers?
- What units are we using to measure? How do those units relate to units others care about (like profit)?
“Until we get brutally honest answers to those questions, I can’t comment on any attempt to measure developer productivity,” Beck said.
Other productivity measurement frameworks released in 2023 adopt a more complex, sociotechnical perspective. This includes, back in May, when the Association for Computing Mastery (ACM) published “DevEx: What Actually Drives Productivity,” which contends that productivity comes from developer experience, a mix of:
- Flow state: developers getting “in the zone” with focused work.
- Feedback loops: time needed for devs to get work done versus time spent waiting.
- Cognitive load: how much a dev needs to know to get that focused work done faster.
Those DevEx metrics expanded research and advice from 2021’s SPACE developer productivity framework, which offers 25 sociotechnical factors to measure.
“In a field where behavior is often measured through technology, it’s important to remember that behind the signals we use to create developer productivity metrics, there are humans at work.”
— “Developer Productivity for Humans,” Google
The newest report from Google, published in December on IEEE, “Developer Productivity for Humans, Part 6: Measuring Flow, Focus and Friction for Developers,” actually aims to measure behavior to drive productivity. This model is grounded in the developer point of view across three areas.
- Developer flow.
- Focus time.
- Friction.
Perhaps indirectly calling out the McKinsey framework, the report reads: “Unlike other measures of productivity, like lines of code or rounds of reviews, the flow and friction metrics we defined are grounded in human judgment rather than assumptions made based on certain tools being used or actions being taken.”
The authors wrote that their research on Google engineers was based on subjective experiences — end-of-workday diaries followed up by interviews — followed by logs-based data, which they validated against the first.
Don’t just lean on the metrics that are easy to gather, like the number of builds, the report recommends. It’s more important — though not as simple — to measure whether devs feel they are reaching that flow state or are stuck on frustrating problems.
Earlier in 2023, Paula Kennedy, co-founder and COO of Syntasso, told us to start off platform engineering roadmap planning by seeing what tickets most commonly appear on Jira — a signal for shared friction and frustration.
The recent Google study allowed developers to define and identify friction for themselves — when they experienced it, what they were working on, and how they resolved it.
At Google, friction most commonly arose around build and test latency, flaky tests and issues with code changes being blocked due to continuous integration failures. It may be very different at your organization, making the diary exercise even more valuable and with low planning.
No matter what way you go about measuring developer productivity in 2024, all roads must start and end with talking to your developers. And with an understanding that it will at least be a little different at every single engineering organization.
And maybe you should frame the whole discussion differently. In the words of Andrew Boyagi, senior technical evangelist at Atlassian, it’s about flipping the question from How should we be measuring developer productivity? to How can we help our developers be more productive?
GenAI for Next-Gen Developer Productivity
Alongside this rise of developer productivity enablement, it’s impossible not to include the meteoric rise of generative AI — or GenAI — in parallel to it. AI will continue to be an important productivity assistant in 2024.
While platform engineering looks to enable teams, much of the early GenAI gains are for the individual. From automatic task approval to AI pair programming, individual task use cases will be key to sustained adoption. Just like with platform engineering, AI assistance for the greatest sources of friction and frustration is key.
Developers, for instance, consistently want better documentation without wanting to write it. In fact, Google Cloud’s ”2023 State of DevOps Report” specifically found that quality documentation can have an estimated 12.8 times more impact on organizational performance.
That’s why the use of case-focused AI tools like Joggr, which looks to use AI to build and update internal knowledgebase inside the integrated development environment, will be very welcome in the new year, alongside the big names.
Just remember that not all generative AI is equal. ChatGPT-generated code is inaccurate 52% of the time — but quite persuasive in its responses.
As the public continues to train these large language models, they will continue to increase accuracy, but engineering organizations must have and communicate generative AI policies to be sure employees aren’t dumping sensitive data into public databases, like what happened at Samsung in 2023.
Not just because of risk to reputation, but because your teams are rapidly becoming used to using these AI assistance tools and won’t readily give up access.
Bringing Context and Copilots to Developers
Developers are quickly adopting tools like GitHub Copilot (92% of U.S. devs took it up in its first six months). Convenience — and devs’ perception that an AI tool will make them more productive — will determine whether they’ll be motivated to give AI-enabled tools a try.
For instance: 85% of dev teams are already working in GitHub, so Copilot, which is already integrated into their repositories, means less context switching. It’s too soon to see the adoption rate of Google’s Duet AI for Developers, just released into general availability, but the idea that it provides natural language assistance across both the technical and collaborative developer experience within the Google Suite is compelling.
Also in December, Atlassian released AI embedded in Jira and Confluence. We will likely see similar AI offerings across other software development life cycles in 2024.
But it’s not just where the chatbots are helping, but the context they bring. Premium versions of generative AI products are not only more secure, as they usually aren’t potentially training the public models on private data, but they are more useful, trained on the specific policies of your company.
The GenAI use cases in 2024 will quickly expand past the individual’s needs to cover more of the SDLC.
One of the most exciting GenAI upgrades that DX CTO Laura Tacho foresees in 2024 is in extracting a written history of the lifetime of an application: “You have an application that is n number of years old, so you have n number of pull requests, of Jira tickets, of design documents in Notion or in Confluence.”
She told The New Stack that “you can train a model on all those things and when a team member is onboarding, starting a new project, debugging something, maybe there’s an outage, AI can be an incredibly effective way to connect with the information that they need much faster.”
But, contrasting with that now notorious McKinsey perspective on productivity, Tacho continued that, “if we look for productivity gains, most of them are outside of the coding task itself, because coding is not 100% of what a developer is spending time on,” which means the intersection of AI and developer productivity will continue to overlap in 2024.
“I think as we get a more sophisticated understanding of where the friction is in the development process, we can apply AI to those problems in very interesting ways.”
Layoffs may still be happening, but most organizations are also still looking for tech talent. Reducing time to onboard will continue to be a goal of productivity teams. The New Stack wrote earlier this year about how platform engineering’s golden paths have been proven to cut Spotify’s developer onboarding time down from 60 days to 20. AI will increasingly be an important tool in that effort.
Rather than waiting for someone to train you — or your new developers— having a conversation with an AI copilot to guide you through learning exercises will make onboarding “more of an active, participatory exercise,” Tacho said.
“AI makes the experience much more sticky for the person and helps onboard them faster.”
But what do you think, The New Stack readers? Post to hashtag #TechWorks where you see — or hope to see — developer productivity headed in 2024.
Learn more about developer productivity and platform engineering from The New Stack’s ebook on the subject. Download it now.
