Software development is expensive, there needs to be a positive ROI to spending extra time trying to make the application take up less space (or use up less processing power), HD space isn't really that big of an issue for most people. If 99.9% of the users aren't going to care if the calculater is 5mb vs 10mb, and aren't going to care if using the calculator uses 5% of their cpu instead of 10%, there's no ROI in spending a time trying to improve that metric.
Example #1 Video games, people will notice FPS drops and there are more visible costs associated with large downloads, so there is some effort spent there on optimization.
Example #2 A web application hosted in the cloud, if we are building an application and it's going to cost us an extra $.10/month for cloud storage if the application is double the HDD size, there's no point in paying ANY attention to that.
As always, money is the language of resource allocation. Developer time being a resource, if the return doesn't outweigh the cost, it isn't laziness that stops people from focusing on optimizing, it is prudent allocation of resources.
In example 2, they do actually attempt to optimize the code to some degree. At least the larger orgs that depend on that stuff, say AMEX. Speed of back end processing is important as is the amount of data needing to be sent for each transaction. Multiply these by millions of transactions per month and you end up at the ROI on doing it.
Why isn't it instinctual to produce optimized code the first time, rather than it being a process undertaken at the end?
In programming there is a rule followed by all skilled programmers. "Premature optimization is the root of all evil."
Optimizing code is difficult and frequently makes code much harder to understand and is itself an extremely common source of bugs. Optimizing code takes extra time. You can't optimize everything and developers usually don't know what actually requires optimization. The 80/20 rule almost universally applies. 80% of your runtime will be spent in 20% of your code. 20% of your code will take 80% of your time to develop. So on and so on.
The question becomes, which 20% of your code will you optimize? If you pick poorly you may spend 80% of you time on code which only effects 5% of the runtime overhead. Which means we need to know because our gut is frequently wrong.
Code optimization, concurrency, and parallelism is some of my core background. This is where profiling comes into plays. You have to have it coded to profile it. If you can accurately test it, we can determine which 20% is taking your time. Which tells you where to start digging.
Sadly, in many cases, optimization doesn't always mean tweaking code, it frequently means rewriting algorithms or changing the algorithms used in the current implementation, which in turn can ripple out to every bit of code which uses the code requiring optimization.
Optimization can be expensive. While there is usually a lot of low hanging fruit to improve things, most coders these days don't really know anything about optimization because it frequently requires low level knowledge and understanding of CPUs, cache, compilers, assembly language, and so on. A small fraction of developers know this stuff anymore. Which means you frequently use your most experienced developers to optimize, pulling from other tasks and further increasing costs.
Which feeds into my previous snarky comment about project management. Historically your experienced developers had broad control of what they worked on. These days, with agile for idiots, everything must be accounted for. As a result, experienced developers spend much of their time fixing street shitter fuck ups, bugs, and pulling things together just so it will run without crashing. Which is why languages like c#, java, and python have become so popular. These languages generally make it harder to crash. Though far from impossible.
Great background. I appreciate it.
IMO the biggest factor in optimization in modern software is libraries.
Modern software often results in larger size files due, in part, to the utilization of numerous libraries and frameworks, which have become an integral part of software development. Libraries provide pre-written code, functions, and modules that developers can leverage to enhance functionality, save time, and ensure more stable and reliable code. However, the trade-off is that these libraries can be quite extensive, containing many additional features and resources that a particular software may not utilize in its entirety. This "bloat" accumulates when developers incorporate several libraries, each possibly containing superfluous code and data, thereby inflating the overall size of the software. Moreover, to prioritize rapid development, ease of use, and robustness, modern software tends to bundle together diverse functionalities and features through these libraries, even if only a subset is required for a particular application, contributing further to the expansion of file sizes and resource demands. This phenomenon can be observed across various software applications, from desktop programs to mobile apps, where developers prioritize feature-rich, multifaceted, and user-friendly environments, which often necessitate the integration of comprehensive libraries and framework.
So, if it is really important for a specific user case (perhaps application needs to fit into a very small size), work can be done to reduce reliance on libraries, and ensure only the bare minimum of code is required, however this requires more time investment.
If so many of today's programs rely on libraries, why not share them among programs? I.e. you'd install all the libraries upfront on a new computer so that programs need not ship with their own copies?
Because most computers have at least 8GB of RAM and a SSD that's as fast as RAM for the suckers stuck on 32-bit and its max of 4GB of RAM. The CPU and the paths the data takes across the motherboard are also fast or powerful enough that coders can get lazy and not have to optimize their code.
The fact that 64-bit systems means there's no limit on RAM means code optimization doesn't need to happen. The fact that storage is cheap ($14 for an 128GB SSD - https://www.microcenter.com/product/659866/inland-professional-128gb-ssd-3d-tlc-nand-sata-30-6-gbps-25-inch-7mm-internal-solid-state-drive) means code optimization doesn't need to happen. (((After all, its cheaper to have the end user of your software to upgrade their system, it's time for a new one anyways, that things an antique.)))
Computer components can't continue improving forever, barring quantum... Stands to reason once physical limits are reached the programs will be forced to be efficient again, like the early days.
(post is archived)