Tag: nvidia

The week on AI – December 22, 2024

The opinions are only mine.

Can Nvidia sustain its meteoric rise?

The new Nvidia Blackwell chip appears to be encountering ongoing challenges. Following design flaws that delayed its release, the chip is now facing overheating issues, making the servers less reliable and reducing their performance. Nvidia has requested its suppliers to modify the design of the 72-chip racks multiple times, causing anxiety among customers about potential further delays. And delays may be worsened because large cloud providers need to customize the racks to fit into their vast cloud data centers. It seems Nvidia is facing the same challenges with the smaller 36-chip racks. In the meantime, customers have decided to buy more Hopper chips.

Nvidia becoming a cloud and AI software provider

Nvidia has been quietly building its own cloud and AI software business (Nvidia AI Enterprise) and is already close to generating USD 2 billion in revenues annually. This is not surprising when we know that all major cloud providers (e.g., Microsoft, AWS, Google) are developing their own AI chips to become less dependent on Nvidia. The AI Enterprise suite includes all the necessary tools and frameworks to accelerate AI developments and deployments, including but not limited to PyTorch and TensorFlow for deep learning, NVIDIA RAPIDS for data science, TAO for model optimization, industry-specific solutions, NVIDIA RIVA for speech AI and translation, and much more. But don’t be mistaken, Nvidia is still far behind the major cloud providers and will continue to operate Nvidia DGX, their AI supercomputer, on the infrastructure of its competitors. Does Nvidia have a hedge compared to other big tech firms due to its proximity to AI hardware? Some believe so. Nvidia still has a long way to go before becoming a cloud and AI software business provider, but it definitely has the means to succeed, and that could become another major revenue stream.

Apple moving in AI chips with Broadcom

Apple is working with Broadcom to develop its own AI chips for servers, aiming for mass production by 2026. These chips are expected to be used internally rather than entering the consumer market, highlighting Apple’s effort to reduce reliance on Nvidia and other competitors. This trend mirrors a broader industry shift, as many tech companies seek to create custom AI processors to cut their dependence on Nvidia. However, designing AI chips is a complex undertaking, and most firms continue to rely heavily on Nvidia, with Google being a notable exception. In most cases, tech companies collaborate with chip makers to leverage their intellectual property, design services, and manufacturing capabilities. The deal between Apple and Broadcom seems to be different from other deals; Apple is still managing chip production with TSMC (it seems). Read

Other readings

> A look at why the world’s powers are locked in a battle over computer chips. How will Europe continue to compete against China from an investment perspective? read
> Broadcom chief Hock Tan says AI spending frenzy to continue until end of decade, read
> Perplexity’s value triples to $9bn in latest funding round for AI search engine, read, read about Perplexity here

The week on AI – November 10, 2024

It’s too soon to call the hype on Artificial Intelligence

Predicting the future of technology has always been a challenge. It’s likely that optimists will face disappointment in the short term, while pessimists—some of whom are even predicting the end of humanity—may also end up being wrong. In other words, as per Amara’s law, we tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run. New technologies often take decades to enhance productivity and often follow the J-curve pattern described by some economists. In this pattern, productivity initially declines before experiencing significant growth. As per Carlota Perez, this was true for the industrial revolution (1770), the steam and railway age (1830), and the electricity and engineering age (1870). Carlota Perez sees Artificial Intelligence as a revolutionary technology, not a technology revolution. AI depends on powerful microprocessors, computers, and the Internet. She argues that AI is better seen as a key development of the ICT (information-communication-technology) revolution that started in the 1970s. Read the whole essay here.

To fully exploit AI, new infrastructure will have to be built, new ways of working developed, and new products and services launched. But AI seems to be on a much faster trajectory than any technology in the past, so we might not have to wait for a decade. Read

ChatGPT is competing with Google for search

ChatGPT has introduced search capabilities that will compete with Google and startups like Perplexity. This search feature is directly accessible in the ChatGPT interface. The AI determines when to use the internet and when to rely on its internal knowledge, but users can prompt it to perform a web search. To further enhance its search capabilities, ChatGPT is also developing long-term memory functionalities. Currently, Google’s search results appear to be more accurate. Perplexity seems still better than ChatGPT in how it presents source references. The ChatGPT search feature is not yet available to users on the free plan but it should be available over the coming months. Competition for the search market is definitely on. Use

The battle for the AI stack

Programming GPUs has historically been complicated, but this changed with the release of Nvidia’s CUDA platform in 2006, which abstracts the GPU complexity from developers. CUDA is a general-purpose platform that allows C code to run on Nvidia GPUs, making it easier for programmers to utilize these powerful processors that are necessary for AI. Most AI engineers and researchers prefer using Python, often with libraries such as PyTorch or Google’s JAX. Under the hood, PyTorch operates on CUDA, which runs C code on the GPU. The industry is now exploring alternatives to CUDA: AMD has introduced its ROCm platform (Radeon Open Compute), and Google has released the XLA (Accelerated Linear Algebra) platform designed for its TPU (Tensor Processing Unit) chips. The key point here is that developing Artificial Intelligence relies not only on the chips created by Nvidia, AMD, and others but also on the software platforms that support these chips, which are equally crucial as the hardware. Nvidia is definitely ahead but the the CUDA platform is getting some serious competition. Read

Other readings

> TSMC to close door on producing advanced AI chips for China from Monday, read
> Salesforce to Hire 1,000 People for AI Product Sales Push, read
> Evident AI index for banks, read