Yuan has built custom computer chips to help its artificial intelligence and video processing tasks, and is talking about them publicly for the first time.
The social networking giant held a virtual event on Thursday to discuss its investments in AI technology infrastructure, after disclosing its internal silicon chip project to reporters for the first time earlier this week.
Related investment news
Investors have been keeping a close eye on Meta’s investments in AI and related data center hardware as the company embarks on a “year of efficiency” that includes laying off at least 21,000 jobs and slashing costs.
While it is costly for companies to design and manufacture their own computer chips, Alexis Bjorlin, vice president of infrastructure, told CNBC that Meta believes the improved performance will justify the investment. The company has also been overhauling its data center design to focus more on energy-efficient technologies, such as liquid cooling, to reduce excess heat.
The Meta Scalable Video Processor (MSVP) is a new computer chip that processes video and transmits it to users while reducing energy requirements. “There is no commercial product” that can handle the task of processing and delivering 4 billion videos a day as efficiently as Meta wants, Bjorlin said.
The other processor is the first in the company’s line of Meta Training and Inference Accelerator (MTIA) chips, designed to help with various AI-specific tasks. The new MTIA chip specifically handles “inference,” which is when a trained AI model makes a prediction or takes an action.
The new AI inference chip helps power some of Meta’s recommendation algorithms, which are used to surface content and ads in people’s news feeds, Bjorlin said. She declined to answer who was making the chip, but a blog post said the processor was “manufactured using TSMC’s 7-nanometer process,” suggesting chip giant TSMC is producing the technology.
She said Meta has a “multi-generational roadmap” for its line of AI chips, including processors for the task of training AI models, but declined to provide details beyond new inference chips. Bjorlin declined to comment on a previous Reuters report that Meta canceled an AI inference chip project and launched another that was supposed to launch around 2025.
Since Meta isn’t in the business of selling cloud computing services like Google parent Alphabet or Microsoft, there’s no need for the company to speak publicly about its internal data center chip projects, she said.
“If you look at what we’re sharing — the first two chips we’ve developed — it definitely gives you an idea of what we’re doing internally,” Bjorlin said. “We don’t have to advertise this, we don’t have to Ads, but you know, the whole world is interested.”
Aparna Ramani, Meta’s vice president of engineering, said the company developed the new hardware to work effectively with its homegrown PyTorch software, which has become one of the most popular tools used by third-party developers to create AI applications.
The new hardware will eventually be used to support metaverse-related tasks such as virtual and augmented reality, as well as the emerging field of generative artificial intelligence, which generally refers to artificial intelligence software that can create engaging text, images and videos.
Ramani also said Meta has developed a generative AI-powered coding assistant for the company’s developers to help them more easily create and operate software. The new assistant is similar to the GitHub Copilot tool that Microsoft released in 2021 with the help of AI startup OpenAI.
Additionally, Meta said it has completed the second or final build-out of its supercomputer, called the Research SuperCluster (RSC), which the company detailed last year. Meta uses supercomputers containing 16,000 Nvidia A100 GPUs to train the company’s LLaMA language model, among other purposes.
Ramani said Meta continues to uphold its belief that it should contribute to open source technology and AI research to advance the technological field. The company revealed that its largest LLaMA language model, LLaMA 65B, contains 65 billion parameters and was trained on 1.4 trillion tokens, which refers to the data used for AI training.
Companies like OpenAI and Google have yet to publicly disclose similar metrics for their competing large language models, though CNBC reported this week that Google’s PaLM 2 model was trained on 3.6 trillion tokens and contained 340 billion parameters.
Unlike other tech companies, Meta releases the LLaMA language model to researchers so they can learn from the technology. However, the LlaMA language model was subsequently leaked to the wider public, leading many developers to build applications employing the technology.
Ramani said Meta is “still thinking about all of our open source collaborations, and of course, I want to reiterate that our philosophy is still open science and cross-collaboration.”
watch: Artificial intelligence is an important driver of big tech sentiment