Keywords

1 Introduction

Human beings transform the world through labor, creating civilization, material wealth and spiritual wealth. The most basic creative activity is creating. Design is a purposefully creative activity, which is preconceived, planned by human beings for purposeful creation and innovation activities. It is also an innovative creation with creative system integration. In the long history of human beings, design always follows the pace of industrial civilization and information civilization, also, design is good at transforming the latest scientific and technological achievements into design energy and spawning new design patterns. With the development of the modern information technology, designers began to use computer intelligence to undertake and assist various complex tasks in the process of design activities. But computer can only participate in the process of assisting design expression and presenting design results. In the innovative age of synergy between design and technology, neural network has been popular in the research communities, and has become a huge wave of technology trend for design intelligence, its unique characteristics of knowledge production for the design of intelligent areas to bring more possibilities. This paper provides a comprehensive survey and analysis, focusing on domestic and international research on neural network and design intelligence at present. Method Based on the extensive literature research, this paper analyzes the research progress of contemporary design intelligence driven by neural network. Results on the basis of current literatures related, this paper sums up the intelligent process of design based on neural network technology, three types of design intelligence and their typical application cases. It also briefly describes the intelligent design tool based on neural network technology. Moreover, it reviews in detail industrial application of neural network in intelligent design. Finally, this study highlights the existing problems and challenges in the field of design intelligence and discusses future development prospects. Hope it is possible to help design researchers and design workers to apply neural network technology to enhance the design intelligence better in the future.

2 The Development of Design Intelligence

With the development of the modern information technology, designers began to use computer intelligence to undertake and assist various complex tasks in the process of design activities. The first stage is computerization, in which the designers present the drawing process by the aid of advanced computer techniques. So designers can separate themselves from handwork. For example, computer drawing software such as CAD can assist designers to draw shapes through preset menus. The computer can not only draw the two-dimensional image, but also establish the three-dimensional space model. With the improvement of computing speed of computer hardware and the further development of software functions, computer aided design has entered the second stage: computerization. At this stage, the design software freed designers from simple repeated operations. Parametric visual programming can adjust input parameters intuitively and change a series of related results in real time through calculation method [1]. This stage indicates that design has come into early phases of digital and intelligent era.

Since 2006, neural network technology is rising again, and is widely concerned by various fields. Today it has become a wave in the field of design. Driven by neural network technology, computer-aided design has entered the third stage: intelligent design. As a new dimension in the field of design, neural network technology owns the characteristics of gaining knowledge. This feature will create a new look to intelligent design and will challenge the paradigm of traditional design practice and academic research, for the machine has began to mimic human intelligence, began to learn, and even got their own “idea” and “inspiration”.

For example, the Chinese character style transfer model developed by Flipboard software engineer Yuchen Tian uses a top-down neural network with CNN architecture to learn and design new Chinese character fonts, so as to realize the transfer of Chinese character styles and transform standard Chinese fonts into target fonts. Paints-Chainer developed a line mapping model based on CNN neural network, which allows for automatic coloring of black and white illustrations and allows for different illustration styles. Georgia tech’s robot Shimon can analyze music in real time and collaborate with humans to improvise music. CycleGAN neural network technology can learn how to convert the characteristics of an image to another image without a double data, including the transformation between horse and conversion, landscape in different seasons. Besides, researchers are currently exploring how to apply this technology to other fields; ChAIr is a design project based on GAN, which can assist artists to carry out more creative design. It uses chair data set for training, and finally gets a model that can generate various chair images, so as to give human designers semi-abstract visual hints.

Current research results show that the neural network technology is accelerating the iterative evolution, more and more professional researchers join the explosion based on the neural network technology. The previous design paradigms are dying out, meanwhile the new design paradigms are gradually rising. How to combine it with design to promote the development of future design intelligence will be an important tendency in the future design field.

3 Introduction to Neural Networks

3.1 The Development of Neural Networks

The neural network is inspired by human understanding of brain biology - the interconnection of all neurons. In 1943, McCuloch and Pits proposed mathematical models of MP neurons. In 1958, the first generation of neural network single-layer perceptron was proposed by Rosenblat. The first generation of neural networks were able to distinguish basic shapes such as triangles and squares, which made it possible for humans to invent intelligent machines that can truly perceive, learn, and remember. The basic principles of a generation of neural networks are limited. In 1969, Minsky published the Perceptron Monograph: Single-layer perceptrons cannot solve the XOR problem. In 1986, Hinton et al. proposed a second-generation neural network, replacing the original single fixed feature layer with multiple hidden layers. The activation function uses the Sigmoid function to train the model using the error back propagation algorithm, which can effectively solve the nonlinear classification. In 1989, Cybenko and Hornik et al. demonstrated universal approximation: any function can be approximated by a three-layer neural network with arbitrary precision. In the same year, LeCun et al. invented a convolutional neural network to identify handwriting. In 1991, the backpropagation algorithm was pointed out to have a gradient disappearance problem. For more than a decade, research on neural networks has been shelved. In 2006, Hinton et al. explored the graph model in the brain, proposed an autoencoder to reduce the dimensionality of the data, and proposed to train the deep belief network in a pre-trained manner to suppress the gradient disappearance problem. Bengio et al. demonstrated that the pre-training method is also applicable to unsupervised learning such as self-encoders. Poultney et al. use energy-based models to effectively learn sparse representations. These papers lay the foundation for deep learning, from which deep learning enters a period of rapid development. In 2010, the US Department of Defense’s DARPA program funded a deep learning program for the first time. In 2011, Glorot et al. proposed the ReLU activation function, which can effectively suppress the gradient disappearance problem. Deep learning has made a major breakthrough in speech recognition. Microsoft and Google have used deep learning to reduce the speech recognition error rate to 20% ~ 30%. It is the biggest breakthrough in the field in 10 years. In 2012, Hinton and his students reduced the Top5 error rate for ImageNet image classification problems from 26% to 15%, from which deep learning entered the outbreak [2]. As shown in Fig. 1, the blue marked points in the figure represent important turning points in the development of neural network technology, and the gray marked points represent the important development period of neural network technology.

Fig. 1.
figure 1

The history of neural networks (Color figure online)

Full size image

4 The Application of Neural Networks

Neural networks have experienced two waves from shallow neural networks to deep learning. There are important differences between deep learning models and shallow neural network models. Shallow neural network models do not use distributed representation and require artificial extraction of features. Deep learning breaks through the limitations of shallow neural networks. As a feature learning method, deep learning can transform raw data into higher-level, more abstract expressions through some simple but non-linear models. Very complex functions can also be learned with a combination of enough conversions. From a model perspective, current deep learning includes: DBN, CNN, RNN, DLRL, DNN.

With the breakthrough of training algorithms and computational capability bottlenecks (especially for the use of graphics processing and high-performance computing), deep learning is widely used in artificial intelligence-related fields, and has made great progress on many research issues. Typical application scenarios include image processing (image classification, object detection, video classification, scene analysis and shadow detection), speech understanding (speech recognition, prosody prediction, prosody prediction, text-to-speech synthesis), natural language processing (syntax analysis, Machine translation, contextual entity linking) and data mining (sentiment analysis, information retrieval) [3]. For details on neural networks please see the review articles.

5 Neural Networks Drive the Evolution of Intelligent Design

Driven by neural network technology, machines begin to imitate human intelligence and learn to learn, with “ideas” and “inspirations” exclusive to machines. The subject of Design Intelligent has the characteristics of neural network technology: high autonomy, high adaptability. These features are well reflected in the design processes, types of Design Intelligence and design tools. At the same time, the mainstream neural network technology platform builds an ecosystem with the help of the open source model, supporting the industrial application in the design field.

6 The Design Intelligence Process Based on Neural Network Technology

Based on the design process theory of academician Youbai XIE and the characteristics of design intelligence, a new process based on neural network is developed (Figure 2). The design process is divided into three stages – the design task of the proposed stage, conceptual design stage and structural design and detailed design stage [4]. Through its own learning ability, the neural network integrates all the processes and becomes an intelligent design subject which automatically gives design output based on sensory input.

Fig. 2.
figure 2

The Design Intelligence process based on neural network technology

Full size image

In the proposal phase of a design task, it usually involves the identification of design requirements (including potential requirements) and analysis. The traditional user modeling system focuses on the psychological state of users, and lacks the behavioral orientation of users, such as interaction preference. So it is unable to give timely feedback on the dynamic behavior, preference and psychological attitude of users. Neural network technology can just make up the defect of traditional user modeling, and user modeling is regarded as a dynamic learning process. User knowledge can be acquired by constantly acquiring user behavior, and user assumptions can be obtained more reliably based on the observation of several applications, and they can be applied to multiple applications [5].

On the conceptual design stage, that is, after the design task is defined, the selection stage of design method needs to be carried out. In the traditional design process, designers and researchers will select new design combination methods based on their own experience and case study, which is relatively inefficient for the new design problems constantly arising. However, the research shows that the neural network can bring rich conceptual design results for different design types and problems. For example, in terms of color matching, based on semantic information transfer technology, neural network can generate a large number of completely different but beautiful and reasonable color matching for the same line draft in a short time [6]. In the field of 3d product design, neural networks accelerate the development of generative design by learning the constraints between physical attributes and acquiring professional knowledge. For example, a four-legged helicopter designed by Autodesk needs both good flight performance and strong load capacity, which requires the helicopter to have a light chassis and low aerodynamic resistance. Unlike traditional design processes, machines learn physical constraints and explore possible structures, generating design concepts that human designers cannot imagine.

On the detailed design stage, based on the ability of neural network technology to learn continuously, the machine can test, evaluate, optimize, backtrack and redesign based on the continuous calculation and real-time update of online data. This makes the detailed design more relevant to the conceptual design phase. In addition, the neural network can also help explain the design results. For example, Airbnb uses the neural network technology to explain their pricing model [7] [8].

7 Three Types of Design Intelligence and Typical Application Cases

Driven by neural network technology, data sources in the field of design intelligence are very complex, including visual data input, that is identification of visual characteristics, auditory data input, that means conversion from speech to text by recognizing sound fragments of information, such as songs, body movement data input, such as Microsoft kinect and Leap motion, which can use neural network to recognize the user’s motion and three-dimensional physical environment. In addition, user behavior can be used as abstract input data, which can be interpreted by neural network technology. Any data that can be converted into electronic signals can be used as input to the neural network [9]. In the design process, there are three types of design intelligence based on different use angles of data: data-driven design intelligence, data-informed design intelligence and data-awared design intelligence. (Figure 3)

Fig. 3.
figure 3

The relationship between data-driven, data-informed, data-awared design intelligence

Full size image

Data-Driven Design Intelligence

Data-driven design intelligence is the direct use of collected data for design decisions. When all the design investigations have been completed, design problems and objectives have been clearly defined, design decisions aimed at specific design details can be made directly based on the results of data collection. A/B test is to obtain users’ preferences by providing different alternatives of different products to different users, so as to directly find the design scheme that can yield better effect. Of course, be wary of “micro-optimizations” or “local maximizations” due to over-reliance on data. When a user creates a site using The Grid, the site asks the user to define the content intent first, then automatically adjust or make changes based on the user’s content intent, and then add personalized content based on the user’s needs.

Data-Informed Design Intelligence

Data-informed design intelligence refers to taking the data result as an input in the design decision-making process. This is because in many cases, the data analysis results are not completely clear to the decision suggestions of design intelligence. In this type of design intelligence, the data output results are used to enlighten designers on how to view design problems, and help to carry out the next design iteration and in-depth research [10]. For example, Facebook often updates its homepage, even if some indicators become worse, it will not affect them to constantly try to change their products, because the fluctuation of data will not affect their design philosophy. Style AI is to use the neural network technology to capture the inspiration source in life and combine the elements of life, nature and art to help designers explore the inspiration source of various fashion from life for creation.

Data-Awared Design Intelligence

Data-aware design intelligence emphasizes that the design process is a process of innovation, not just design decisions made from instances of data collection. In the design process, what kind of data types and combinations of types need to be obtained is a design problem in itself. Compared with other two types of design intelligence, design intelligence based on data perception is a more strategic way of thinking. Google set up a large number of data centers, most of these data centers are deep learning of the neural network set up by the CPU and GPU, through the data center, data designers and scientists need to work with developers and business strategists, design system actively, in order to collect the right data types or data type combination to solve problem [11]. As shown in Fig. 4, the three types of design intelligence are briefly summarized.

Fig. 4.
figure 4

Three types of design intelligence and typical application cases

Full size image

7.1 Design Tool Based on Neural Network Technology

Design tools change as design objects change and technology evolves. RoelofPieters and SamimWiniger analyzed the development history of computers and summarized their three maturity levels as human design tools (Fig. 5): first generation assisted creation system(AC1.0), it can simulate the tools in digital form; then is second generation assisted creation system(AC2.0), in this stage people and machinery can work in harmony through real-time feedback loop; and third generation assisted creation systems(AC3.0), negotiate the creative process fine-grained, augment creative capabilities and accelerate skill acquisition time [12].

Fig. 5.
figure 5

The development of the generation assisted creation system

Full size image

Intelligent design tools driven by neural network technology have evolved into the third generation. The strong autonomy and adaptability of neural network technology endow the intelligent design tools with the ability to constantly optimize the design results, and redefine the man–machine collaboration mode: collaborative production, human–machine collaboration and mutual enhancement. Neural network technology can help illustrators draw by modifying strokes, give writers assistance on writing by improving text styles, and help musicians compose music by proposing ideas. At present, the design intelligent tools based on neural network have presented two main types: research type and industrial application type. (Figure 6)

Fig. 6.
figure 6

Design tool based on neural network technology

Full size image

Research-based design tools

Research-based design tools can make it easier for ordinary users or developers to understand the neural network technology, stimulate the interest of users or researchers, and collect a large amount of training data to continuously improve the neural network technology. Google’s AI Experiments are designed to create cool and interesting products using cutting-edge technologies such as neural networks, so as to explore the possibilities of neural network technologies on a larger scale. Its Autodraw allows users to draw pictures at will, and then, based on the understanding of the patterns drawn by users based on the neural network, vector icons drawn by professional design staff can be matched for users to choose. Users can then paint on this basis and draw to complete their designs. (Figure 7) AI Painting developed by IDEA Lab of Zhejiang university is a collaborative creation platform between AI and human, which enables users to express the imaginary world into reality. (Figure 8) The user can use the AI Painting system to create animated stories out of imagination and experience, and create beautiful animated landscapes with just a few strokes of paint.

Fig. 7.
figure 7

AutoDraw Google

Full size image
Fig. 8.
figure 8

AI Painting system

Full size image

Industrial application design tools

The progress of neural network technology supports the research and development of innovative tools in the field of design. At present, design tools based on neural network are being widely applied in visual design, clothing design, product design, architectural design and other fields, especially in the realm of visual design.

Neural network technology penetrates visual design industry gradually, which accelerates the development of visual design industry and realizes manpower change into brainpower. Traditional visual design tools output the content of the projected onto a two-dimensional plane space through the communication between the system and the technologists. However, visual design tools driven by neural network can extract the inherent law hidden in the back of data through deep learning and extends more possibilities through reinforcement learning. Through the disassembly and abstraction of a large number of excellent case design patterns, a set of design and development standards, workflow and related tools that can intelligently configure the brand language are constructed. Alibaba’s “Luban” system – one-click generated banner advertising system is a typical case, which constructs a set of self-optimizing design process from understanding layer composition to aesthetic and commercial evaluation system through machine (Figure 9).

Fig. 9.
figure 9

Alibaba Luban system

Full size image

The development of neural network technology promotes the transformation of traditional digital clothing design, which is the inevitable trend and result of the new type of industrialization and informatization of the clothing industry. Deep learning algorithm is also applied to recommend personalized fashion collocation for users, help users to match clothes, and generate personalized clothing collocation for users by building a framework to learn semantic information about visual style. That is Thread (Figure 10). In addition, Project Muse, an artificial intelligence clothing design product launched by Google, supports users to hand-draw and generate personalized fashion dress (Figure 11).

Fig. 10.
figure 10

Project Muse page

Full size image
Fig. 11.
figure 11

Thread’s fashion advice

Full size image

The development of neural network technology also promotes the transformation of products design. For instance, Zoomorphic Design proposes a way to create morphing shapes by combining artificial and animal shapes. To identify a pair of shapes that are suitable for merging, the Zoomorphic Design team uses an efficient kernt-based technique (Figure 12). The merging process is formulated as a continuous optimization problem in which two shapes deform together to minimize the energy function combining several design factors. The modeler can adjust the weights between these factors to gain a high level of control over the resulting shape. Thus, it is ensured that the morphing shapes do not violate the design constraints of the artificial shapes. Zoomorphic Design demonstrates the versatility and effectiveness of this method by generating various morphing shapes [13].

Fig. 12.
figure 12

Zoomorphic designs created by the system

Full size image

The application of neural network technology in the architectural design industry is deepening, which not only improves the work efficiency, but also makes the characteristics of architectural design better reflected. Gen Nishida et al. put forward the 3D modeling model of architecture applied by neural network: according to the sketch sketched by the user, the system automatically generates 3D architectural model. Based on deep learning, the system finds a set of predefined components that match, then the user selects the appropriate components from the set of components. The system synthesizes the user’s options and generates a 3D model of the building [14]. In the future, enhancing the application of neural network technology in architectural design is the inevitable choice for the architectural industry to get rid of the traditional operation mode and move towards intelligent design.

In the future, under the change of new design mode, the design tool based on neural network will become a more widely used tool and an important assistant tool for designers. Neural network endowers tools with recognition ability and natural interaction ability, which will reduce the threshold of using design tools and learning costs, and greatly improve the design expression efficiency of end users.

8 The development prospect of design intelligence driven by neural network

8.1 Problems faced by neural networks in the field of design intelligence

Neural network technology still faces many challenges in solving design problems. This paper mainly discusses and analyzes from the two main aspects of technical basis and design thinking:

  1. 1.

    Based on the technical level: the rise of the neural network technology was largely due to the wide availability of huge amounts of data. At present, the complex data related to design has growth exponentially and is present through a variety of different morphology (such as text, images, audio, video, etc.), meanwhile it has different distribution. All of this poses challenges to the application of neural network in design. Therefore, how to design effective neural network model and learning theory based on big data, and gain exponential knowledge from the exponential growth of data is now an inevitable challenge in the development process of intelligent design driven by neural network.

  2. 2.

    In terms of design thinking: neural network technology is best at solving problems with clear objectives and rules, such as reasoning, classification, proof and clustering. However, the design is characterized by unclear problem definition, uncertain solution path and unclear evaluation criteria. Design thinking has three characteristics. First, design thinking is a process of cross-media knowledge processing; second, design thinking is the coevolution of problem solutions; third, the whole design process is divergent and convergent. On the whole, the design presents the characteristics of first divergence and then convergence. It can be seen that the problems that neural network technology is good at solving are in conflict with the characteristics of design thinking, and there are many challenges in how to solve design problems through neural network technology.

9 Future Development Prospects

The neural network has brought a disruptive change to the field of design. The understanding of “neural network-driven intelligent evolution of design” should not be limited to the technological basis, production organization and lifestyle changes, but the system and management changes at a deeper level. The future development of neural network technology may include:

  1. 1.

    At the technical base level: neural network reconstructs the design data stream and drives the design industry to become technology-intensive. Driven by the neural network, the data-intensive paradigm of design has come. A large amount of unstructured data is generated in the design and production process: consumer data, industry data, design data, designer data, etc. The collection, processing and intelligence of these data is the key to enable design to access data economy. In the future, driven by neural network technology, designers’ design works will be precipitated into design data, and the connection between people and data will be realized by constructing knowledge graphs. After the connection of massive data, semantic data and image data will greatly improve the production efficiency of the design industry. Technologies such as neural networks and big data have transformed the design industry from a labor-intensive industry with a lot of low-end repetitive work to a technology-intensive one.

  2. 2.

    At the level of production and lifestyle: neural networks have revolutionized the design process, driving mass customization and human–machine collaborative creation. Designers are no longer using user sampling to find requirements and provide a solution; Instead, it involves itself in the whole production life cycle of the product and continuously adjusts and optimizes the product in the actual experience of users. Under the new design mode, design is no longer a node in the production chain of products, but an important link in the user-centered design cycle endowed with artificial intelligence, which runs through the whole product life cycle [15]. Neural network technology is a multiplier of efficiency in the process of modern production and life. “digital, intelligent and customized” manufacturing and design have become an important feature of the field. After taking data as the intelligent guidance of computing, mass customization will become the main production organization mode. Design intelligence can realize that results grow and reproduce on their own according to customer’s own design intelligence data. Future everyone to get the design result can be different with others, and in the process of the use of the products, based on neural network algorithm of real-time computing user experience, continuously adjust and optimize the product, so as to realize the mass customization of personalization, redefining the man–machine collaboration mode of collaborative production, man–machine coordinated and mutually reinforcing.

  3. 3.

    At the level of management system reform: the neural network realizes the de-elitism of design, drives the role change of designers and the reform and transformation of design education. With the development of neural network technology, contemporary creation-making activities are greatly different from the past in terms of concept and technology. In particular, with the further development and popularization of micro-manufacturing, users can be more deeply involved in the design and generation of artifacts, and the completion activities are transferred from elite entrustment to co-creation. In the design process, users not only provide their own personal views and design knowledge, but also participate in the generation process, showing a trend of acting as part of the role of product design implementer. At the same time, neural network technology will become an important auxiliary tool for designers, and designers will no longer only play the role of traditional product design implementer, but show a trend of gradual transformation to the role of rule definer of product generation. Creativity, the ability to recognize opportunities to deal with complex problems and the ability to think critically will become the core competence for the development of designers. In the era of intelligent design, design education is faced with reform and transformation. It is necessary to upgrade design education and promote interdisciplinary cooperation, so as to cultivate a new generation of design talents with innovative thinking ability.

10 Conclusion

Design always follows the pace of industrial civilization and information civilization, also, design is good at transforming the latest scientific and technological achievements into design energy and spawning new design patterns. At present, neural network technology is vigorously driving the rapid development of design intelligence from design process, design tools, design expression and other aspects, which makes the development of design intelligence enter a new stage. In the future, if the problems faced by neural network technology can be overcome, neural network technology will bring more possibilities to the field of design, and the intelligent ecology of design will be more efficient.