Lately, when it comes to AI in software development, the talk revolves around tools like ChatGPT, which generate lines of code, making the work of millions of developers worldwide easier. Another reasonably popular approach is integrating artificial intelligence components with existing software models. This involves, for example, adding an AI model to a smartphone responsible for facial recognition or other functions. Today, I will approach this topic from a slightly different perspective and introduce two methodologies for software development – Software 1.0 and Software 2.0.
Software 1.0 is eating the world
Software 1.0 is the most popular paradigm for software development. Developers can access various tools such as programming languages, compilers, linkers, libraries, and other similar resources. By understanding how to use these tools, we can create a set of operations for our hardware’s processor, which translates into specific actions. Most software operates by building strict rules for managing a particular simulator, such as autonomous vehicles, where we create very specific rules to control the simulator.
Taking the example of autonomous vehicles, if we press the gas pedal, meaning we perform an action, the system executes predefined lines of code related to the principles of dynamics and physics. This approach is often called “Software 1.0 is eating the world,” meaning that software collects information that humans already know about the world, consumes it, and converts it into instructions that describe it. This can be applied to the operation of any device equipped with an integrated circuit, such as a car, an elevator, or a microwave.
However, a significant problem arises here. In most cases, even though we can describe the control of a car in a relatively autonomous manner, our model, which builds reality, is incredibly complex. It consists of thousands of mathematical equations and many factors and uncertainties related to the environment. A minimal change, such as in wheel traction, leads to a change in assumptions because different physical forces start to operate, and the dynamics are altered. Of course, as programmers, we can describe all of this with successive mathematical actions, but it would take an enormous amount of time. Additionally, maintaining such code would be impractical, mainly due to the inevitable occurrence of errors in such complex projects.
Software 2.0 is eating software
Thanks to AI, we are witnessing a significant technological leap, especially in the field of software development. This is primarily due to the new approach called Software 2.0, created by Andrej Karpathy, one of the founders of OpenAI, who currently serves as the AI director at OpenAI and previously held a similar role at Tesla. The ideology of Software 2.0 is about replacing parts of the code in projects described in the previous paragraph with artificial intelligence elements. Using the example of losing traction on a car tire, the neural network in our AI model will take environmental data from sensors or cameras and approximate a specific action to be taken.
In other words, in Software 2.0, instead of manually creating and implementing mathematical equations into the software as in Software 1.0, programmers can approximately describe all phenomena and cover a much more comprehensive range of events using AI rather than doing it themselves. The AI architecture will solve about 90% of the problems that we would describe with, for example, 100 mathematical equations primarily related to wheel movement and the physics of contact with the ground. Additionally, this approach simplifies code maintenance, which is typically the most expensive part of the development process.
Comparing Software 1.0 and Software 2.0
In the 1.0 approach, if we have developed software for a device and need to make changes because we’ve replaced one of the electronic components with another with different parameters, it means that all the mathematical models we created related to the parameters of that component are outdated, and we must discard them. When applied to a large project, such as building an autonomous vehicle, there are thousands of such elements that affect the final product. In contrast, in the 2.0 approach, this is not a significant problem. Rebuilding the software involves retraining our model with new data related to the component and implementing the new model in place of the old one. This is much easier faster, and also allows for code maintenance to be eliminated.
Additionally, it might be more beneficial for the end product, as the new component may have better parameters that open up new possibilities. For instance, by installing a higher-resolution camera in our vehicle, our neural network receives more accurate environmental information, resulting in more precise processing. Another advantage of Software 2.0 is code optimization for performance, aiming to relieve the processor’s memory. In this case, we can, for example, reduce some parameters from our recurrent neural network, decreasing the memory requirements. Subsequently, we retrain the network using techniques like knowledge distillation. Of course, this will slightly reduce the efficiency of our solution, but our goal will be achieved relatively quickly.
Such optimization is a highly time-consuming and challenging process in Software 1.0, as demonstrated by computer game developers who create their products for diverse platforms. In the era of Software 2.0, when designing applications for various platforms that support different programming environments, the entire process becomes more efficient. We no longer need to specify in detail how each microcontroller or GPU should process data. Our task is to optimize and deploy our neural network models to be compatible with a specific operating system and hardware. We no longer need to worry about low-level communication and CPU interpretation. The key is implementing the right Software 2.0 framework at the right time and ensuring that the processor receives appropriately prepared data for computations.
Limitations of Software 1.0, Transformer Architecture, and Fine-Tuning
Another challenge with the Software 1.0 approach is its susceptibility to significant software changes. For instance, if we have an autonomous car and have written instructions on moving forward and stopping independently, our software is adapted to all possible variables (various types of cameras, surfaces, weather conditions, tires, engine versions, loads, etc.). If we wanted to add a feature to turn only to the left in our perfect system, we would have to rebuild the entire system because the system’s dynamics change entirely. We would need to consider even more variables, such as the wheel’s turning angle, the gas pedal’s angle, or traction variables.
The complexity of calculations increases exponentially! In this case, it is easier to discard the entire code and start from scratch rather than modify it, but this would prolong the project’s timeline. In the automotive industry, it’s unsurprising that many projects last up to 15 years! Of course, Software 2.0 works differently. Thanks to relatively recent technology, the Transformer architecture (used by ChatGPT), we don’t have to start from scratch but only fine-tune the model with fresh data to introduce new functionality. Returning to our example, this would be data describing the left-turn manoeuvre, such as driver camera data or synthetic data created by us.
We can do the same with any additional feature. This process is called fine-tuning and involves teaching the system a new skill. This is precisely what we can observe in the case of ChatGPT. It consists in teaching AI to formulate statements the way specific people do. In Software 2.0, we teach our AI model in a simulation to perform a particular task, providing it with high-level feedback – whether it did it correctly or not. It will eventually learn to complete the task perfectly, better than any human. In Software 1.0, we program and teach specific tasks, primarily through coding and mathematical descriptions.
Software 2.0 – The Cybersecurity Challenge
Software 2.0 is built on neural networks with a certain number of parameters. The effectiveness of neural networks is somewhat measurable. However, there may be situations where information is manipulated. For example, if we change one pixel in a camera image that AI analyzes, the system may fail to recognize the image correctly. This problem does not occur in Software 1.0 because the images that the system needs to recognize are described simply and precisely.
The system always reads them correctly. In Software 2.0, such cases, mainly related to cybersecurity incidents called adversarial attacks, do happen. It is incredibly challenging to test and detect these issues during development. Specialized software is required to test the system thoroughly. Simulators that introduce noise into the data processed by AI to train the system to perform perfectly in situations where the data deviates from the ideal are the solutions. This is highly complex and costly to develop, but it solves most problems. Companies like Tesla and SpaceX are pursuing this path.
How Billennium adopts Software 2.0
Billennium also aims to follow the direction of Software 2.0. This involves not only creating new products or solutions for our clients but also suggests omitting specific mechanisms in existing software. An example of this could be choosing ChatGPT and replacing search engines in applications or corporate systems. This will allow for more accurate responses based on discovered correlations and training the model on received data and user expectations.
Additionally, we offer our clients modifications in the software we have developed and continue to maintain. This primarily makes future tasks, such as system maintenance, making changes, adding features, integrating with new products or other systems, more accessible and shorter. After modifying the software to the Software 2.0 form, the next step is simply providing the data on which we conduct fine-tuning processes.
Learn more about our AI & Data and Software Development Competence Centers.
