Friday, December 03, 2021

Reckless Software Construction

Dear software developers: please please please stop making disposable software.

Demand for rapid development of software has radically changed how we develop, deploy, and maintain software over the past decade or so.  Storage is cheap.  Bandwidth is plentiful.  Agile and rapid iteration allow fast-to-market delivery.  All of this leads to fast, inefficient, messy, and reckless Rube Goldberg software construction.

I want to be clear: I am not trash-talking all software engineering via component composition (where devs take pre-built things and slap them together to make a new thing).  This is useful for many service-based products or web applications where the software runs on the software company's own hardware.  This strategy is over-used and is not the tool for every job.  I like some quality DevOps and do love a good web app, but the problem is when software is thrown together without care and diligence.

The equation is simple: moving fast and cutting corners leads to increased risk.

Agile dev + fast-to-market + prefab software = unnecessary risk

Stop it.  Be an engineer, not MacGyver.

"Our CISO will take care of this."

No.  Your CISO doesn't want to remediate your security problems after-the-fact.  Your CISO is there to help guide your development efforts down a safe path, NOT to compensate for your desire to "ship fast, fix later".  You should be working with your security team to avoid unnecessary risk and not call them in as a clean-up crew when you screw it up.  

Fast to market is great, but success will be short-lived if you are also fast-to-hacked.

Security teams are increasingly tasked with securing what some call a Software Supply Chain: the third party, often unknowing, contributors to a software product.  This chain of components (or frankly, pre-packaged #includes, libraries, or services) creates risk that needs to be at least monitored.  All of the third-party things your devs use to deploy your product will potentially fall victim to the next SolarWinds, hacked NPM package, or Kaseya.

Take Inventory

The provenance of software has become so out of control that even the White House is urging an inventory of all the stuff (a Software Bill of Materials, or SBOM) you use to make your things.  Cybersecurity risk is high not only because attackers know hacking your stuff is profitable, but also because your stuff is too complicated.  There, I said it: the software you created is unnecessarily complicated. This call for SBOM is a cry for help; you need to at the very least know what your stuff is made of.

Take Responsibility

This need for SBOM is a symptom of developers’ rush to pass the responsibility for fair and safe software onto others.  So you make a thing out of other peoples' things... it is YOUR responsibility to vet the components.  Your customers are relying on you to do this!  Requiring an SBOM help someone else vet your product, but it's not foolproof and is not for end-users. Software developers must take responsibility for securing their work and not blindly build trash out of other peoples’ trash. SBOM is not a panacea, it is a stopgap until we regain control over the junk we build.

Take Ownership

Ultimately when your software is compromised, your customers will blame you.  Will you pass the blame to your third-party suppliers?  You should be building something you're proud of and something you UNDERSTAND.  The right tool for the job is not always the fanciest/newest tool.  You do NOT need to use docker containers to build a solitaire app on iOS. 

Take Action

The answer is not "more software."  The answer is not "hire a consultant."  The answer is not SBOM. The answer is not DevSecOps (though that can help).

The first thing you should do is seriously think about your engineering practices... like really think about it.  If you develop client-side software (apps), evaluate the components and tools you use to build your app.  Maybe throwing it together quickly got it out the door, but how much effort are you spending maintaining it?  Are you building the right thing?  Phone Apps are bloated.  Why is my email app 270MB?  Why does this email app update EVERY DAMN WEEK?  These are questions I'm asking your developers, and you should too.  Doing your job well requires more than just tossing the product over the wall and going out for a beer.

Consider factoring out third-party components when you only use a small part of the component.  Do you really need jquery and d3.js and Angular? Do you need all those NPM packages?  An elegant, well-crafted architecture results in a more stable and secure product.  Take pride in your design and not just your speed. Focus. Keep it simple as you can; any amount of early investment to understand and reduce complexity of your software's provenance will minimize the chance that someone else's mistake ruins your product.

9 comments:

Michael Hamrich said...

I agree totaly. Do you mind if I use that qoute and point to your blog on my LinkedIn
post I am plannong to write? Constand Delivery Deployment is a buzz word that many non technical excutives / managers seem to LOVE to use. Without any concept of reality!

The equation is simple: moving fast and cutting corners leads to increased risk.

Agile dev + fast-to-market + prefab software = unnecessary risk
Stop it. Be an engineer, not MacGyver.

Sid Stamm said...

@Michael Hamrich - Of course! You're welcome to quote, attribute, and link to my post.

Frank Lancaster said...

I think this is a great post and I agree with many of the points, but I'm not sure if the people reading this would be in a position to do anything about it. There will always be pressure from upper management to get software out as quick as possible to build the highest margins possible. If this means not researching all of your dependencies, or not documenting the dependencies properly, those will be the first to go.

Personally, while I get wary of government overreach in some areas, I think the biggest motivating factor is money. Large fines for releasing unsecure software and securing it later (if ever) is the only good way to counter this practice in my opinion. I know that solution isn't perfect, and I definitely agree we've entered a new world where security needs to be a top priority.

Sid Stamm said...

@Frank - regulation is not the only answer; pressure on management from tech leads or project managers can also work. If you can't make changes for the better, share this with those who can!

Unknown said...

If security isn't mandated by external entities in a uniform manner across the marketplace then many companies will cut security to reduce costs and increase their short-term competitive advantage. That decision comes from upper management.

Employees at the level of project managers or tech leads can't fundamentally affect these sort of changes.


Sid Stamm said...

@Unknown 7/12/22, 5:58 PM: I agree that the corporate incentives are to cut costs not directly impacting the bottom line, but upper management doesn't make decisions in a vacuum -- at least not at well-managed companies.

PMs and tech leads should be seen as expert leaders within the company and should feel compelled to build safe software. Good managers/execs will listen to them and at least consider their arguments. Bad managers/execs won't listen, and I encourage tech leads and PMs to identify this behavior as a warning that their employer might be stifling their impact instead of helping them build their career.

Anonymous said...

This is enforced upon engineers by managers via tech leads. The individual engineers have little say unless if they are okay with risking getting fired.

Sid Stamm said...

@Anonymous wrote: The individual engineers have little say unless if they are okay with risking getting fired.

This depends heavily on the organization and work environment; good management and leadership doesn't happen in a vacuum (just like I said above). "Shut up and do your job" is a terrible way to lead, especially in today's job market.

Engineers looking for career advancement will try to show vision and affect the direction of products they work on. Good managers (and tech leads) will try to embrace this "passion for the cause" and won't fire the ones eager to do more than they're told.

IcyCold said...

Engineers looking for career advancement will try to show vision and affect the direction of products they work on. Good managers (and tech leads) will try to embrace this "passion for the cause" and won't fire the ones eager to do more than they're told.

Completely agree. To effect more adoption of good practices by engineers visibility into the problem is crucial. I believe SBOM will serve as a foundation to help visualize and alert programmers about choices that were made, something that i find missing. An automated SBOM production process at build time along with an sbom consumption tool, with alerts triggered by policies(security team/ managers / legal) and at affordable prices(dont need an entire security suite), would really help developers iterate on their choices.