The duality of tech debt and vendor debt
Why I say "no" to most vendors even if I believe their product would improve my tech stack
There are many parallels between accumulating vendor relationships and accumulating internal technology. At the bottom of this post I’ll list out some of those parallels with pithy one-liners. My inspiration for this post is my experience of building out a modern ML stack, and I expect this is relevant to many other people actively doing their best to pick an LLM stack as their company tries to adopt more generative AI.
In the ML ecosystem there is an almost uncountable number of companies providing services. To name but a few capabilities: data storage, feature management and computation, pipelines, environment and package management, model development environments, artifact management, inference deployment, and model monitoring. Generative AI has custom challenges in all those areas, and adds in new ones like serverless foundation models and LLM fine-tuning. Large cloud providers (Amazon, Google, and Microsoft) all try to have full-service offerings, while there are also startups specializing in each specific element of the stack, and other startups that cover a variety of subsets.
Identifying which companies would be best for each part of your ML stack is hard enough on its own, let alone figuring out which combination of them will work together without too many hiccups. There are just so many permutations. This problem becomes even more complex when you add in a temporal element, where those providers change their offerings over time and your own needs change too.
A number of companies, from small to large, have reached out to me to sell their offerings. Often for new founders I’ll gladly give feedback or explain my circumstances and the current challenges with my stack; in those situations I’ll manage expectations about whether I’m interested as a buyer. But for other reach-outs, my most common answer is essentially a polite “No”. No to migrating, no to a free trial, no to a demo. It’s never personal, and I do sympathize with people building great products and trying to build a company in this difficult sales environment. It’s often not the right time for me to explore new options for part of the ML stack, regardless of what the seller can offer to make it easier.
Sometimes, you can’t even give your product away for free.
Here’s why
I don’t mean to imply that I’ve at all solved the ML stack that I need, or even that I’m happy with the status quo. Just that I need to judiciously manage, and only selectively grow, that set of products. A big part of that is the effort to manage the technical complexity of using those products, but another large part is managing the vendor relationships. Every vendor product has overhead, and the combined overhead can be a drag on a company and its engineering organization.
Adding a new vendor might sound easy. Sometimes it’s as simple as creating an account and adding a (company) credit card. Yet, “easy” does not mean that you aren’t creating future problems for yourself and your company. You might need to allocate budget and review that budget annually. You may lose access, either from not storing your credentials properly or by not noticing mandatory migrations to better authentication. Your data might end up used in different ways than you expect, leading to a difficult discussion with your company’s privacy and/or legal people. The vendor could increase their fees without you noticing. Your usage might organically increase to a threshold where you are forced into an enterprise pricing tier. The vendor may change their API or even deprecate the features you use.
This pattern, where consequences happen later on and aren’t always obvious ahead of time, reminded me of managing tech debt.
Tech debt is the burden of technical work that you accumulate for your future. That might be work and problems left behind for your future self, or for your eventual successors.1 That includes the bugs you must eventually fix, the dependencies you will have to upgrade or migrate, and the surface area you will have to document and support.
With thoughtful and prudent technical accumulation, you can reduce the amount of tech debt you accumulate. Typically, the more functionality you build, and the faster you release it, the more tech debt you leave behind.
Vendor debt is the burden of technical and organizational work that you accumulate for your future from the vendor relationships you have. This includes the contract renewals that will need to be signed, negotiations for new terms, billing and payment, the bugs you will file, the feature requests you will make, timing adoption of new features or migrating aspects like their API or authentication methods, tracking and managing your administration permissions as people join and leave your company, security lapses from the vendor, regular questions about what user data you share with the vendor and the privacy of that data, the marketing and cross-selling you’ll be subjected to, tracking spend internally and explaining overspending (or underspending, for that matter, as an unused budget becomes a juicy target to fix another area’s overspending), and possible sale of the vendor or even closure of their business. These are some of the burdens that will fall to you, your legal team, your finance team, your procurement team, and anyone else involved in managing the vendor.
With thoughtful and prudent vendor accumulation and integration, you can reduce the amount of vendor debt you accumulate. Typically, the more vendors you use, and the faster you add them, the more vendor debt you leave behind.
Dualities between tech debt and vendor debt
Tech debt: Free libraries come with few promises and may become deprecated
Vendor debt: Free service tiers come with few promises and may become deprecated
Tech debt: Unsure about your historical commitments to internal clients
Vendor debt: Unsure about your historical commitments in old contracts
Tech debt: Accumulated clients that are hard to track or that have ownership changes
Vendor debt: Reliance on vendors where you are unsure why you use them or how exactly you incur costs
Tech debt: Maintain your software. Communicate roadmaps, track bugs, renegotiate feature requests…
Vendor debt: Maintain your vendor relationships. Communicate needs, follow up on feature requests, renegotiate terms, renew contracts…
Tech debt: Every layer in your stack comes with risk for your reliability and security
Vendor debt: Every vendor in your stack comes with risk for your reliability and security
Tech debt: With enough tech you’ll invest in systems, processes, and people to track and manage the backlog
Vendor debt: With enough vendors you’ll invest in systems, processes, and people to track and manage the relationships
Tech debt: Reuse existing functionality instead of rewriting or building something new and shiny
Vendor debt: Reuse existing providers instead of adding a new and shiny one
Tech debt: If possible, don’t start two large migrations at the same time
Vendor debt: If possible, don’t onboard two large vendors at the same time
Tech debt: Decline to build something new in the first place; don’t out-run your maintenance ability
Vendor debt: Decline to add new vendors when you’re struggling to manage existing ones
Tech debt: Purge old code
Vendor debt: Purge old vendors
Tech debt: People have more energy for creating messes than they do for cleaning them up
Vendor debt: People have more energy for creating messes than they do for cleaning them up
Tech debt: The right amount of tech debt is more than zero
Vendor debt: The right amount of vendor debt is more than zero
“Successor” is a funny word in this context. Those are people who succeed you, who come after you, but without you necessarily setting them up to succeed.