Heterogeneous Hardware Needs Universal Software

As hardware advances and diversifies, we are entering what many see as a new golden age for computer engineering. However, the idea of ​​ever expanding hardware can be daunting for software developers, because while hardware versatility is great for price and innovation, it can lead to software complexity on the back end. There are so many hardware-specific software tools out there running on this world where CPUs, GPUs, FPGAs, ASICs, DSPs, and much more mix freely; An invitation to interoperability headaches and escalating technical expenses. The question on developers’ minds is how to navigate this ever-changing ecosystem to take advantage of the new world of hardware acceleration without driving themselves crazy in the process.

The challenge for developers is that if we look historically, every time a new piece of world-changing silicon appears, it often comes with a new software package of tools. With heterogeneous hardware innovation moving at an already almost untrackable pace, keeping up with the end of software is starting to feel untenable. In response, a new philosophy emerges: we should be able to have a common set of tools, libraries, frameworks and assemblers that can handle a variety of accelerators in a “silicon-neutral” way.

Anyone doing DevOps business today should be asking, “Are the tools we are adopting ready for the future of heterogeneous devices?” Instead of using exclusive proprietary stacks of individual vendor concepts of how acceleration works, we should think about layers of abstraction that allow us to move to open software suites that work across multiple vendors and infrastructures.

This philosophy is central to a number of movements – one example is an API – to demand the most diverse libraries, compilers, and analysis tools. It is imperative for any future-conscious developer to analyze the tools they use and ask whether they are preparing for the future or locking them in one solution that excludes you from other innovations.

When it comes to navigating the changing developer landscape, it’s especially important to take a look at the multi-vendor puzzle piece. One of the distinguishing strengths of computing and electronics is their ability to interact with each other. Without cross-platform compatibility, we end up being trapped in individual solutions that don’t take advantage of the diverse landscape of innovation at our disposal. If systems are built from the ground up with mix-and-match in mind, we don’t have to change our software tools to adapt to new technologies and can use the same languages, libraries, and analysis tools we’re accustomed to.

It’s important that we take on this work now because not only is the hardware diversity at an all-time high, its momentum is only growing. In the past when the hardware landscape was more limited, switching to different toolkits for different CPUs was possible, if not an elegant solution. However, computers today have computing capabilities from multiple vendors. If my tools don’t extend across all of their ecosystems, I’ll end up in a complex mess pretty quickly.

A particularly relevant example can be found in the field of supercomputing, one of the most exciting and fast-moving areas of innovation in the developer community. What is the most exciting thing about supercomputing? It’s heterogeneous, allowing multiple architectures to take on specialized hardware acceleration for the different tasks they’re best suited to. The Aurora supercomputer, for example, is planned to achieve up to 2 exaflops of performance—a staggering number (the first supercomputer I helped design had 1 teraflops—or 2 millionths of Aurora’s performance). The amazing capabilities of these devices are only possible due to advances in heterogeneous memory technologies, storage capabilities, and high-end CPUs and GPUs all working together seamlessly.

The advances we see in supercomputers, as always, will eventually carry over to lower level systems. In the near future, no matter what kind of work you do, massive computing power will come online. We need to make sure that our libraries and tools can take full advantage of them when they arrive.

There are two great places to learn more about activities to help software be open, multi-vendor, and multi-engineered for heterogeneous programming: https://sycl.tech/ And https://www.oneapi.io. There are many tutorials online, link to SYCL book (free PDF Download) and much more.

Leave a Comment