This article discusses the development path of AIGC technology -> products -> business applications and value realization, and explores how AIGC can achieve closed loops and value landing in the industrial sector through industrial examples.
Sequoia Capital’s predictions for AIGC related product maturity
Considering the industrial applications of AIGC, such as in manufacturing and construction, the content produced by AIGC cannot be limited to images and text; it needs to enter the more information-rich three-dimensional domain. Next, we will explore the development path of AIGC technology -> products -> business applications and value realization, and discuss how AIGC can achieve closed loops and value landing in the industrial field through industrial examples.
1. AIGC Technology: From Text to Images
From the increasing number of tests on ChatGPT, we can see that ChatGPT can not only parse and structure semantics but also perform data analysis based on this using NLP (Natural Language Processing).
ChatGPT’s structured processing and data analysis – provided by Jiage Data
In fact, a number of AI drawing frameworks or platforms led by Stable Diffusion had already caused a stir earlier last year. Although images appear to contain more complex information than text, their technology matured earlier than text generation led by GPT. It is necessary to revisit how these image AIGC frameworks work, using the mainstream open-source framework Stable Diffusion as an example.
Images generated by Stable Diffusion already exhibit abilities comparable to human artists
Stable Diffusion mainly consists of three components, each with its own neural network.
1. CLIP for Text Encoding: It composes a 77*768 matrix of semantic information output from text. CLIP trains AI to simultaneously understand natural language and perform computer vision analysis. CLIP can determine the correspondence between images and text prompts, such as progressively matching the image of a building with the word “building”. Its training capabilities are achieved through over 4 billion images with text descriptions globally.
CLIP’s training dataset
2. UNET and Scheduler: This is the famous diffusion model main program (proposed by CompVis and Runway teams in December 2021, the “Latent Diffusion Model” (LDM)), which is used to predict noise to achieve the reverse denoising process, thus generating images in the information space. As shown in the image, the dye diffusion process resembles the gradual transformation of an image into noise, and when researchers add random noise to the image, AI learns the overall process in reverse, thus obtaining a model for generating images from noise in the information space.
The reverse denoising process of the diffusion model
To explain with a simple example, if some dye is randomly dropped into clear water, over time, it will create beautiful shapes like the one shown in the image. Is there a way to reverse engineer the initial amount of dye, the sequence, and the initial state of the water tank based on a specific time and state? Clearly, without AI, this is nearly impossible.
Different dyes create different shapes in water
3. Decoder from Information Space to Real Image Space: This converts the matrix information in the information space into visible RGB images. Imagine the process of communication; the sound signals we hear are converted into text signals that our brains can understand and stored in our minds. This process is called encoding. If we attempt to express the text signals in some language, this process can be called decoding – the expression can be in any language, each corresponding to a different decoder. Decoding is merely a method of expression; fundamentally, it is based on the human brain’s description and understanding of a certain matter.
Full process interpretation of StableDiffusion from input to output
It is precisely due to the connection of these key technical steps that Stable Diffusion successfully created an all-powerful AI drawing robot, which not only understands semantics and converts them into information flow in the information space but also creates through simulated denoising in the information space and restores it into visible images through the decoder. This sci-fi-like process, when viewed in a world without AI, is nothing short of miraculous.
2. AIGC Technology: From Images to 3D Models
While image generation has achieved breakthrough results, if these results can be further optimized and applied to more fields, they may realize greater value. We have also seen exploratory results in some niche fields, such as achieving better control over image generation through scene understanding and adjusting parameters with different datasets, rather than merely obtaining better results through trial and error with text.
2.1 Design Intent Generation
At the beginning of 2019, the “This Person Does Not Exist” series generated by GANs garnered significant attention overseas. In China, we have also seen companies launch results in niche fields. The team also experimentally launched the “AI Creative Library” on mobile in August 2022, where simply inputting a sentence allows the dialogue robot to quickly understand the semantics and generate multiple images closely aligned with architectural concept plans within a minute. Furthermore, by inputting an existing image and modifying some descriptive keywords, the “AI Creative Library” can generate a series of derivative images to assist designers in finding inspiration in their daily creations.
Little Library Technology’s “This Building Does Not Exist”, GANs model generating architectural imagery and iteration process
Left Image: Generated by Little Library’s “AI Creative Library”, triggered by the statement “Louis Kahn style, a small museum by the water”; Right Image: Generated by Little Library’s “AI Creative Library”, based on the left image of Louis Kahn style, completing the style switch to Le Corbusier
To optimize the effects of the “AI Creative Library”, the team has made some new explorations: since existing algorithms and models are more focused on general internet materials, the data reserves for architectural-related images, descriptions, and styles are evidently insufficient. Here, a special identifier for architectural-related vocabulary was adopted to create a fine-tuned prior dataset, which was then integrated into training to enhance the model. Through the newly enhanced model in the architectural professional field, the quality rate of the test set compared to the original model improved by 13.6%.
Google Dreambooth Fine-Tuning Algorithm Illustration
For example, when inputting a museum image and the term “Zaha Hadid” (the globally renowned female architect who passed away), the model can understand that it needs to align the museum’s architectural style or features with Zaha Hadid’s works, rather than adding a character or image of Zaha Hadid in the museum, or creating a cartoon image of Zaha Hadid in the AI world – which is often one of the results returned by general models.
After fine-tuning, the architectural model generated by Little Library’s “AI Creative Library” can fully understand the implied meaning of the special term “Zaha Hadid”.
2.2 3D Model Generation
While two-dimensional images are impressive, their application in industry is currently limited to the role of “conceptual image libraries”. In the future, if they are to become precise expressions of design results, they need to advance towards 3D and higher information dimensions.
In 2020, when AIGC was not as mature as it is now, the aforementioned team was exploring how to use AI to generate 3D models. They publicly shared their algorithm for generating models from graphics in the DigitalFUTURES workshop at Tongji University, showing that the model’s effectiveness at that time was not ideal. However, it was valuable in achieving the linkage between graphics – images – models.
Results from the 2020 DigitalFUTURES workshop at Tongji University, where the Little Library teaching team achieved graphics to images to model generation
The following year, in the DigitalFUTURES workshop teaching at Tongji University, the team released an algorithm that learns the relationship between satellite images and real 3D models through GANs, generating real 3D models from satellite images. This algorithm can roughly restore the 3D stretched shape of the main objects corresponding to the satellite images by performing feature learning on different layered elements in the satellite images, predicting the height of the original object corresponding to the projections of different objects. However, this method still has certain limitations; it can only be used in satellite image scenarios, making it difficult to accumulate similar relationships between other scene images and 3D shapes. Additionally, the restored 3D shapes can only roughly predict height, and other details need to be regenerated through algorithms, leading to significant discrepancies with real 3D models, limiting its application scenarios to early project assessments.
Illustration of layered feature extraction training for urban 3D models
Results from the 2021 DigitalFUTURES workshop at Tongji University, based on GANS for reconstructing 3D models from satellite images
Thanks to the explosion of AIGC algorithms and the increasing maturity of 3D generation algorithms, we are also seeing vertical AI companies begin to absorb more advanced technologies and ideas to improve their models, with new exploratory directions emerging in the 3D-AIGC space. For example, OPENAI launched the Point-E framework, which can predict any 2D image as a point cloud through algorithms, and then predict 3D objects from the point cloud.
Illustration of the entire process of the PointE framework
However, the quality of the generated models still has certain limitations, primarily reflected in the following three aspects:
1. Difficulty in restoring 3D shapes: First, 2D image data appeared earlier than 3D model data, and currently, the available 2D image data is also more abundant than the latter, so the former can serve as training material at a larger scale. The limited generalization ability of the scarce 3D model training material makes it difficult to restore the original 3D shape;
2. Overall material deficiency: The most critical aspect for 3D models is the filling and selection of materials; however, the method of deducing materials directly from images is not yet mature. Similarly, the performance of materials varies under different shapes, environments, and light sources, making it nearly impossible to achieve material reconstruction when these variables are concentrated in a single image;
3. Inaccurate precision of generated models: Models deduced from point clouds typically rely on the density of the point cloud to reconstruct the object’s surface mesh. If the point cloud is too sparse, the object will be severely distorted, or even impossible to reconstruct.
The Little Library team tests the Point-E model; the left shows a building image generating point clouds, which in turn simulates the right 3D model. Unfortunately, the result is just a bunch of meaningless point cloud models, and Point-E still cannot comprehend an image of a building.
Of course, we can understand the current technical bottlenecks. If we slightly lower the goal and choose to generate simple shapes from 3D modeling software, taking 2D screenshots to reconstruct in the Point-E model, we often find that the results are better than those above tests, but still limited to the category of “preliminary drafts.” This is closely related to the training set; generating 2D views from various angles using 3D modeling software is one of the easiest methods for this model to obtain training data.
The Little Library team tests the Point-E model, selecting a simple 3D model from modeling software to take arbitrary angle screenshots for reconstructing the 3D model, often achieving decent results.
In summary, the technical route from text -> images -> point clouds -> 3D objects is indeed astonishing, but if it is to be applied in the industrial field, there is still much work for AI scientists to do.
However, is this the only technical route to achieve the generation of 3D models?
3 New Ideas for AIGC Applications in Vertical Fields
In the development of large models across various fields, companies led by OpenAI, including giants like Nvidia and Google, are also launching their own universal 3D-AIGC frameworks, unfortunately still in an early stage. For vertical entities in industry, the practical application evidently has a long way to go.
Globally, in the field of generating 3D models, aside from large universal models, some vertical industries are also exploring how to apply AIGC. For example, Siemens has conducted policy simulations and further optimizations for generated models in engine design and manufacturing, ultimately achieving real-world delivery of 3D model generation results through 3D printing.
Siemens achieves engine design and simulation through generative algorithms
Such achievements depend on the continuous iteration of underlying business content and data standards in the industrial logic.
According to the ISO/IEC definition of digital standards for content, SMART (Standards Machine Applicable, Readable and Transferable): Level 1 is paper text, with no machine interaction possible; Level 2 is open digital format, with very low machine interactivity; Level 3 is machine-readable documents, but machines cannot understand the results and content retrieved; Level 4 is machine-readable content, allowing semantic interaction but machines cannot understand the logical relationships of the context; Level 5 is machine-interactive content, enabling automatic recognition, automatic generation, and other intelligent attributes.
In the industrial field, L3 level information content is widely used, while L4 level digital content is being developed, and L5 level intelligence is the core foundation of Industry 4.0 and intelligent manufacturing. Therefore, generating machine-readable content above L4, especially generating intelligent content at L5 level, is the future direction of AIGC.
ISO/IEC SMART digital standards – Study on the Current Status and Trends of Standard Digitalization Development in China Engineering Science, 2021, Volume 23, Issue 6, Liu Xize, Wang Yiyi, Du Xiaoyan, Li Jia, Che Di
Overseas, practical applications in the AIGC industrial field have already begun, while domestic explorations remain relatively scarce. However, we have also identified some companies deeply engaged in vertical fields, such as the aforementioned Little Library Technology team in the construction industry. We will use its practical work in the construction industry as an example to explore the landing path of AIGC in vertical industries.
The current domestic real economy is in a transitional window, with the national level proposing the important task of “integrating artificial intelligence with the real economy.” Various industries urgently hope that AI technology can truly land and assist industries in achieving digital and intelligent upgrades, rather than remaining as a conceptual DEMO product or an amusing topic of discussion.
The construction industry is a national pillar industry worth nearly 30 trillion yuan annually, but its level of digitalization ranks last among all industries in the country. The current national policy aims to promote intelligent construction, hoping to reach a new level of “Chinese construction.” Intelligent construction is based on new industrialization of construction (industrialization/assembly, digitalization, intelligence), deeply integrating new generation information technology with advanced construction technology throughout all stages of design, production, construction, operation, and supervision, featuring self-perception, self-decision-making, self-execution, self-adaptation, and self-learning. Its aim is to optimize the quality, efficiency, and core competitiveness of the entire lifecycle of the construction industry.
China’s construction industry’s total output value and growth from 2011 to 2021 – National Bureau of Statistics – Qianzhan Industry Research Institute,
Data sources: Gartner; Kable; OECD; National Bureau of Statistics; Bloomberg; McKinsey Global Institute analysis
In the construction industry, the underlying data standards are transitioning from machine-readable documents at L3 level in the CAD era to machine-readable content at L4 level in the BIM era. The requirements for 3D models in the construction industry are that content objects possess comprehensive and precise information in three-dimensional space, including models, data, etc. If they can also include rule dimensions, they can possess intelligent capabilities such as self-perception, self-learning, and self-iteration. Currently, L3 level CAD and L4 level BIM application software have been monopolized overseas, and our development space and potential must focus on L5 level, which can cover low dimensions with high dimensions.
Illustration of content format for digital standards SMART in the construction field
Based on insights into the digital transformation of the construction industry, the Little Library team realized that it was necessary to redefine the underlying data of the entire industry. Since its establishment in 2016, it has been committed to the underlying technology research and development of L5 level 3D model AIGC and its application in the construction industry. Based on an AI system that contains business flow logic, it generates “data-model-regulation” interlinked content that includes building information and multi-dimensional data, 3D models, and rules/regulations/laws to achieve intelligent generation of architectural design solutions.
Illustration of the intelligent cloud model ABC at L5 level
AI Recognition In this field, the team achieved 100% cloud restoration and 99.8%* accurate semantic parsing and supplementation for L3 level non-semantic CAD drawings by cleaning and training millions of CAD drawings of different business types, reaching a world-class advanced level in this field. This achievement has been deeply applied to various products and solutions of the company, such as the “intelligent drawing review” for construction drawing examination, where the accuracy rate of checking clauses is about 96%.
Little Library’s construction drawing components and spatial recognition
AI Analysis In this field, based on effective project recognition, the team can perform physical environment simulation analysis, human behavior data simulation and prediction, and analysis and simulation of big data related to projects for common civil building types such as residential and commercial buildings. On the application level, it can assist clients in quantitative analysis of project plans; for example, by evaluating the entire line of residential products of real estate companies, different value assessment coefficients can be obtained to help improve product quality. Therefore, Little Library Technology was also selected as the first AI judge for the China Real Estate Association’s housing type design competition. This capability has also been applied in the development and operation of more than ten commercial buildings in Hong Kong and domestically.
Little Library’s “Product Value Assessment”
AI Optimization In this field, the team believes that “optimization” is a further iterative pursuit based on the previous “recognition” and “analysis,” i.e., re-generating better results based on existing content. Such technology has been applied in the company’s specific products and solutions. For instance, in the 2022 version of the design cloud “intelligent sunlight optimization” feature, Little Library can automatically adjust plans that do not pass sunlight verification to ensure they can pass without making significant changes to the original layout. This capability is also used in the deepening of architectural design plans; for example, in the curtain wall design optimization scenario of a museum project in Sichuan, in collaboration with the Sichuan Commercial Design Institute, Little Library’s algorithm optimized over 30,000 irregular triangular curtain wall panels into 12 standard modules, reducing the number of possible variations from 116 to 90%, significantly lowering costs for the building’s curtain wall due to reduced SKU and mold quantities.
Little Library’s “Curtain Wall Optimization AI Algorithm”
AI Generation This field is the core part of intelligent design. For the construction industry, selecting economical, practical, and aesthetically pleasing design solutions and delivering safe, efficient, and high-quality construction results require coordination among multiple professions and roles. It is necessary to tackle challenges from macro to micro scales, covering various disciplines such as architecture, structure, mechanical and electrical, plumbing, and landscape, as well as various types of buildings, including residential, commercial, and industrial. Therefore, generating specialized results in vertical fields is not solvable by a single model or algorithm with one set of data; it requires the organic integration of multiple models, modalities, datasets, and business logic, along with product design that fits niche scenarios and continuous iterations based on user feedback to ultimately achieve success.
The Little Library team, starting from business logic, organized the 24 business process steps required for traditional architectural design, extracting and reconstructing the core content into six business modules, establishing a new architectural design AIGC business process centered on AI systems and cloud architecture: Call (information retrieval and AI recognition), Make (full AI generation and human-machine collaborative generation), Modify (manual adjustments and AI optimization), Check (data verification and AI review), Collaborate (cloud-based multi-person collaboration and business management), Output (automatically outputting in more formats – 3D models/2D drawings/images/PPT/Excel, etc.).
Left Image: Original 24-step business process for architectural design; Right Image: Little Library restructured into six business process modules under AI support
Based on a deep understanding of business and the restructured business logic, the product design integrates the six major business modules with technologies such as AI recognition, AI generation, big data, and cloud collaboration, fulfilling various architectural business needs from analysis to design to review, gradually covering the breadth and depth required for residential business.
Little Library Design Cloud – Architectural Planning Product with Six Major Modules
Little Library Design Cloud – Architectural Entity Product with Six Major Modules
4. The Value Landing of AIGC in Industries
In most industries, the application of AIGC is still in its early stages, and the continuous development of overall AI technology will drive further innovation applications of AIGC. Taking the current practices in the construction industry as an example, AIGC can currently assist in enhancing specific business scenarios with high efficiency requirements, such as investment research, design, evaluation, management, and construction in the construction industry.
4.1 Optimal Solution Gains and Efficiency Improvements
In the investment research phase of the construction industry, the “Two Concentrations” policy (centralized land supply and centralized land auctions) introduced in 2021 has led to a large amount of land being released within a month. Development companies need to complete investment assessments for each piece of land in a short time, with the core question being how to find the optimal architectural planning scheme on a piece of land to obtain the highest product value and investment return calculations. Originally, completing a residential planning conceptual scheme would take at least 3-5 days, which cannot meet business needs, thus raising the demand for extreme efficiency in pre-investment architectural planning schemes.
The Little Library team launched an AIGC architectural planning scheme that can output initial plans in about 30% of the original time. More importantly, AI can generate and optimize schemes that humans may not have thought of or that are difficult to arrive at through manual exhaustive reasoning, thus achieving better performance or economic results. For instance, in a project by China Jinmao in Jiangxi, the AI-generated scheme not only took 20% of the time of the original method but also increased the total product value by 56 million yuan compared to the original scheme. In the nine months of the 2021 land auction market, the team completed nearly a thousand projects and tens of thousands of schemes, successfully assisting clients in acquiring numerous land parcels.
AI-generated actual residential land acquisition scheme from “Little Library Design Cloud”
4.2 Cost Reduction and Energy Saving
In the actual construction phase, the Little Library team combined AI with DFMA (Design For Manufacture and Assembly) design methods, collaborating with the construction giant China State Construction Engineering Corporation’s subsidiary, to deeply integrate box-type prefabricated buildings with AI design generation and L5 level ABC “data-model-regulation” linkage, achieving real-time linkage of investment – plans – costs before implementation, reducing design and cost changes by 80%, and effectively lowering the overall SKU and mold quantities of prefabricated components, achieving over 50% energy savings and emissions reductions. In a hotel project in Shenzhen, the team completed the design to construction in four months, significantly shortening the total construction period by at least 14 months and saving over 60% of time.
Full process intelligent design and construction of a hotel in Shenzhen by “Little Library Assembly Cloud” in collaboration with China State Construction Engineering Corporation
Comparison of L5 level intelligent construction model with traditional model
Through the above cases, it can be seen that L5 level AIGC can start from the source of data generation and effectively assist the industrial chain in achieving higher quality, efficiency, and core competitiveness throughout the entire lifecycle through specific applications in segmented scenarios across the industrial chain. In the future, it is an inevitable trend for AIGC to transition from text and images to higher-dimensional 3D and L5 level content results; this is not only the future expectation of the construction industry for artificial intelligence but also a common expectation across various vertical industries.
Note: * On the basis of no obvious errors in layers, the current AI recognition accuracy for standard components (doors, windows, walls, stairs, elevators, air conditioners, fire hydrants, parking spaces) is 99.8% (the test set consists of thousands of architectural floor CAD drawings sourced from several leading developers’ internal standard libraries).
References:
-
The Illustrated Stable Diffusion – Jay Alammar – Visualizing machine learning one concept at a time.
-
Robin Rombach, Andreas Blattmann, et al. High-Resolution Image Synthesis with Latent Diffusion Model (CVPR 2022 Oral)
-
Nataniel Ruiz, et al. DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation (2022)
-
Alex Nichol, Jun H, et al. Point-E: A System for Generating 3D Point Clouds from Complex Prompts (2022)
-
Liu Xize, Wang Yiyi, Du Xiaoyan, Li Jia, Che Di, et al.: ISO/IEC SMART Digital Standards – Study on the Current Status and Trends of Standard Digitalization Development in China Engineering Science, 2021, Volume 23, Issue 6
-
“Digitalization Level of Various Industries in China” – McKinsey Global Institute
This article is sourced from the internet, Copyright Notice: Only for articles pushed by the “Huaxia Qihang Research Institute”
Long press to follow the QR code to download resources for free
Previous Recommendations:
Invitation to the National University AIGC Teaching Innovation and Business Application Faculty Training Course
Notice on the Launch of the Advanced Training Course for “Virtual Reality Application Technology Talents”