Korean startup Rebellions' latest artificial intelligence chip ATOM / Courtsey of Rebellions

In an office building located in southern Seoul, a dozen chips are laid side by side on shelves, Monday afternoon, each next to its own electric fan to cool it down as it operates.

These chips, called ATOM, are the latest neural processing units (NPUs) developed by Korean startup Rebellions, targeting AI models with up to 7 billion parameters. Their performance is being tested and compared with a few of Nvidia's A100 GPUs, located in a separate room on the same floor.

Regarded as the next generation of AI chips, NPUs are processors optimized for simultaneous matrix operations, meaning they are a step further in deep learning compared with general-purpose central processing units (CPUs) and graphics processing units (GPUs).

Rebellions, the fabless AI chip startup co-founded by five Korean engineers in 2020 to focus on AI inference as opposed to training, was viewed as the best hope to match or rival Nvidia in that field.

Park Sung-hyun, CEO and co-founder of Rebellions, told The Korea Times in an interview earlier this week that ATOM is set to be mass-produced with Samsung's 5nm technology in the first half of 2024.

That will be an important milestone for the Korean chip industry, as ATOM will the first domestically developed chip to support language models at mass production level.

"We are much more energy-efficient than Nvidia's GPUs in AI inference," said Park, referring to the process of running live data through a trained AI model to make a prediction or solve a task. ATOM is up to five times more power-efficient than Nvidia's A100, with just half of the latency — a measurement of speed for chips — of Nvidia A2 in a language model inference benchmark test, he added.

More visually, the ATOM chips only need fans for cooling down, while Nvidia chips need to operate in an air-conditioned environment — which means more power consumption and higher operating costs.

Rebellions' second-generation AI chip ATOM is being tested in the company's office in southern Seoul, March 15. Courtsey of Rebellions

The rise of Rebellions comes as the global semiconductor industry is in a heated battle for AI chips, fueled by the sweeping popularity of openAI's ChatGPT and ever-increasing demand for Nvidia GPUs to provide the massive computing power needed by LLMs.

South Korea, known for its position in memory chipmaking, is poised to gain a strong foothold in this booming market. Samsung Electronics, for instance, is facing off with TSMC to challenge its dominance in AI chipmaking, while the Korean government aims to grow the share of locally developed AI chips in domestic data centers to 80 percent by 2030.

Rebellions had secured backup from several biggest names in the Korean tech industry — including Samsung, KT and Kakao.

In January, Rebellions finished its latest funding round totaling $124 million, led by KT, which has so far invested over $50 million in the startup. That brought Rebellions' valuation to $650 million, becoming the most-funded chip startup in the country.

Park said going into mass production will be a big boost for the company's revenue, following years in the prototype stage. KT, the second-largest telecom operator and largest data center company in Korea, will become the first customer of Rebellions after it begins mass production of ATOM, according to Park, who believes telecom companies are the most ideal clients for AI chips.

"The future of the AI industry is going to be an infrastructure game. Telecom companies are willing to invest large amounts of money into building up data centers," he said.

Park also hopes the partnership with Korean's largest data center company will act as a reference for its global expansion, especially in the U.S. market. The company has been in talks with some major U.S. hyperscalers such as IBM, Park said.

Park, who graduated from MIT in 2014 with a Ph.D. in electrical engineering and computer science, had spent six years developing chips in the U.S., working for companies including Samsung's U.S. research arm, Intel and SpaceX.

Despite his experience in the U.S. chip industry, Park saw better chances for starting a semiconductor company in Korea. "In the semiconductor hardware ecosystem, Asia is going to be the next big thing," he said. "South Korea and Taiwan are catching up."

Park Sung-hyun, CEO and co-founder of Rebellions / Courtesy of Rebellions

Park went back to his homeland and founded Rebellions in 2020, joined by Oh Jin-wook — another Korean AI chip expert with an industry background in the U.S. — along with other co-founders.

For Park, it was not an easy decision, as most of his industry network at that time was in the U.S. Rebellions' initial team members included figures from IBM, Intel and Apple, as well as Samsung USA.

Now, it has around 120 employees including nearly 100 engineers. As the team expanded over the years, it attracted experts with local background, with those coming from Korean big semicon companies making up nearly 70 percent of the staff, according to Park.

Rebellions launched its first product called ION in 2021, targeting AI tasks in the financial industry with chips manufactured by TSMC. Then the company decided to refocus on chips for AI models, after seeing higher demand from data center companies instead of finance companies.

Samsung, the largest chipmaker in Korea, is Rebellions' manufacturing partner for ATOM. The two companies deepened their cooperation last October by announcing plans to co-develop REBEL, the startup's next-gen NPU after ATOM, which is set to target LLMs with larger parameter size and compare with Nvidia's top-spec GPU H200 in inference tasks.

Park is pinning larger hopes on REBEL, which is expected to finish development and start mass production in the second half of this year, using Samsung's 4nm fabrication process.

ATOM, Rebellions' second-generation NPU, targets AI models with up to 7 billion parameters with a single chip. Courtesy of Rebellions

As the generative AI market continues to expand, Park expects the demand for inference chips to become a key growth point.

"AI training traffic is depending on the number of developers, while AI inference traffic goes up as number of end users increases," Park said. "Currently, training and inference market is almost half and half, but inference traffic is going to be much higher in the future."

Within two and a half years, Park saw the company's share in the Korean LLM market grow to 30 percent in terms of inference tasks, and take up "at least 3 to 5 percent" globally. Currently, Nvidia is estimated to have a whopping share of up to 90 percent in the global AI chip market, according to analysts.

"If you catch just a little share of the LLM market, that's already huge," Park said.

Ann Cao is a tech reporter with the South China Morning Post. She is currently based in Seoul, reporting for both The Korea Times and the South China Morning Post under an exchange program.