Huawei Helps National Library of Norway Turn Cultural Heritage into AI Ready Data

The National Library of Norway is using high performance storage infrastructure from Huawei to transform its extensive digital cultural heritage into a resource for developing AI technologies. At the heart of the solution is Huawei OceanStor Dorado All Flash Storage, deployed within the library’s on premises AI environment.

The National Library of Norway has a legal mandate to collect and preserve publications produced in the country. Its collection includes rare books dating back to the 16th century, historical newspapers, radio and television recordings, as well as modern digital publications and audiovisual content.

For more than 20 years, the library has operated a large scale digitization program. The process goes far beyond scanning and archiving and includes OCR processing, image correction, file format normalization, quality control, and metadata generation.

To date, the library has collected around 20 PB of unique digital objects, while the total volume including protection copies reaches approximately 60 PB. To protect this information, the library follows a 3+2+1 data protection strategy, maintaining three copies across two different storage technologies, with one copy stored offsite.

As artificial intelligence continues to develop, this historical data is gaining additional value. Norway has around five million Norwegian speakers, two official written standards, and numerous dialects, creating specific requirements for training AI models that accurately reflect the language and cultural context.

The Norwegian Ministry of Culture and Equality has therefore entrusted the National Library with developing its own large language models. The initiative receives annual funding of NOK 70 million, while the library also benefits from long standing agreements with publishers that allow copyrighted content to be used for model training.

This is where Huawei OceanStor Dorado All Flash Storage plays a key role. The solution provides more than 2 PB of capacity for retrieving data from the archives and performing intensive processing tasks such as data cleansing, deduplication, formatting, and validation.

With high throughput and low latency, the platform enables large volumes of historical data to be processed more efficiently. This helps reduce storage bottlenecks and allows computing resources to be used more effectively during AI dataset preparation.

After processing by CPU clusters, the prepared datasets are transferred to Sigma2, Norway’s national e infrastructure, where they are used to train AI models.

The project demonstrates how modern storage infrastructure can connect two very different environments – long term cultural heritage preservation and high performance AI data processing. While the archival environment is optimized for durability and long term retention, Huawei’s all flash platform provides the performance required to actively process and prepare data.

For the National Library of Norway, this approach transforms decades of digitization efforts into a foundation for next generation AI applications that can better reflect the Norwegian language, culture, and history.

ASAP and Huawei – Partnership for Modern Infrastructure

Huawei is a partner of ASAP, with the two companies working across modern IT infrastructure, storage solutions, networking, and AI technologies. Through its partnership with Huawei, ASAP provides access to high performance storage and infrastructure solutions designed for organizations with demanding requirements for reliability, performance, and scalability.

The deployment of Huawei OceanStor Dorado at the National Library of Norway demonstrates how modern storage technologies can help transform large digital archives into AI ready infrastructure and create a practical connection between cultural heritage and future AI applications.