Automated sequence and motion planning for robotic spatial extrusion of 3D trusses
 115 Downloads
Abstract
While robotic spatial extrusion has demonstrated a new and efficient means to fabricate 3D truss structures in architectural scale, a major challenge remains in automatically planning extrusion sequence and robotic motion for trusses with unconstrained topologies. This paper presents the first attempt in the field to rigorously formulate the extrusion sequence and motion planning (SAMP) problem, using a CSP encoding. Furthermore, this research proposes a new hierarchical planning framework to solve the extrusion SAMP problems that usually have a long planning horizon and 3D configuration complexity. By decoupling sequence and motion planning, the planning framework is able to efficiently solve the extrusion sequence, endeffector poses, joint configurations, and transition trajectories for spatial trusses with nonstandard topologies. This paper also presents the first detailed computation data to reveal the runtime bottleneck on solving SAMP problems, which provides insight and comparing baseline for future algorithmic development. Together with the algorithmic results, this paper also presents an opensource and modularized software implementation called Choreo that is machineagnostic. To demonstrate the power of this algorithmic framework, three case studies, including real fabrication and simulation results, are presented.
Keywords
Robotic spatial extrusion Sequence and motion planning Digital fabrication1 Introduction
The advancement of additive manufacturing offers unprecedented means for achieving geometrical complexity for continuum structures at the architectural scale. However, conventional layerbased 3D printing techniques, such as fused deposition modeling (FDM), are not appropriate for threedimensional discrete truss structures because of the anisotropy structural behavior incurred by the material deposition process (Fang 2017) and prohibitively long fabrication time. In contrast, robotic spatial extrusion, sometimes called spatial 3D printing, has been proven to be an appealing alternative to fabricate bespoke 3D trusses design (Hack and Lauer 2014; BranchTech 2018; Tam et al. 2018; Yu et al. 2016). This technique involves extruding and solidifying thermoplastic along prescribed linear paths in space to form spatial meshes or lattices, taking advantage of the precision and flexibility offered by industrial robots.
However, despite the advantages of the accuracy, efficiency, and the new fabrication possibilities offered by these multiaxis machines, the level of automation in this designfabrication workflow is still comparably low. While transitioning between a digital design model and machine code for a 3axis gantry machine is straightforward, for multiaxis robots, gaining fine levels of control and bypassing the complexity of generating collisionfree robotic trajectories is much more nuanced and subtle, which requires significant effort.
Existing investigations in the field of architectural robotics often involve manual planning of path guidance for the robot’s end effector, followed by tedious diagnosis for potential problems in a trialanderror manner. This slow and cumbersome workflow deviates from the initial purpose of having such a digital designfabrication workflow: to forge a smooth and direct transition from digital design to realworld machine materialization; instead, the current process often requires a complete reprogram for the robot whenever the target geometry has a small change. This technical challenge in the sequence and motion planning and programming of the robots congests the overall digital design/production process and often confines designers to geometries with standard topology with repetitive patterns. Thus, to broaden the topological class of roboticprintable truss structures, an automated sequence and motion planning system is needed.
This work presents a new algorithmic framework for robotic sequence and motion planning in the context of spatial extrusion. The planning framework is implemented as a flexible planning tool, called Choreo, that allows users to input unconstrained spatial trusses and receive an automatically generated feasible extrusion sequence and robotic trajectories. Choreo communicates with designers in the language of geometry and topology, while shielding the users away from the computational complexity of sequence and motion planning. It fills in the critical link between digital geometry and machine fabrication, playing a similar role of slicing algorithms in the layerbased 3D printing context. Case studies in simulation and on real hardware are presented to show Choreo’s power in enabling automated planning for robotic extrusion of complex 3D trusses with nonstandard topologies, which has not been shown possible before.
1.1 Contributions and organization of paper

This paper embodies the first attempt in the field of architectural robotics to formally formulate sequence and motion planning problem (SAMP) in the spatial extrusion context, using the language of Constraint Satisfaction Problem (CSP).

This paper presents a new hierarchical planning algorithm to solve the SAMP problem in an integrated, streamlined algorithmic workflow for the first time. By rigorously formulating the problem and presenting a new planning algorithm as a baseline, this research sets up a corner stone for future work on the algorithmic improvement.

Inside the planning hierarchy, the sequence planner solves a relaxed extrusion planning problem using a backtracking search algorithm with userguided decomposition and constraint propagation; empirical results are presented to compare the tradeoff between several strategies to assist the search and show the computational overhead induced by the extrusion planning problem, which differentiates it from the general discrete CSP problems.

The novelty of the motion planning module includes a new samplingbased semiconstrained Cartesian planner to enable existing motion planners to work with long planning horizon.

A modularized and customizable implementation of the proposed planning framework is presented. The planning software, called Choreo, is adaptable to various hardware setups and can be smoothly fitted into existing digital design workflow.
2 Related work
In response to the need for automated planning in robotic extrusion, this section summarizes previous efforts in the area of robotic extrusion and planning. Key research from five distinct fields, (1) architectural robotic spatial extrusion, (2) path planning for architectural robotic assembly, (3) computer graphics, (4) manipulation planning, and (5) task and motion planning is presented, with contributions and drawbacks highlighted. The aim of this section is to demonstrate why an integrated planning system, which combines features from all the above fields, is needed for robotic spatial extrusion to be fully accessible to architectural designers.
2.1 Architectural robotic extrusion
Robotic extrusion (sometimes called spatial 3D printing) involves extruding a thermoplastic along linear paths, typically to form a mesh or grid structure, using robotic motion. In recent years, this idea has been presented as an alternative to layerbased additive manufacturing for discrete spatial trusses, with advantages both in terms of mechanical properties (Tam et al. 2018) and speed of construction (Oxman et al. 2013; Hack and Lauer 2014; Mataerial 2018; BranchTech 2018). However, the flexibility of the industrial robots has mostly been deployed to facilitate complexity in shape (as opposed to topologies): morphed grids with repetitive zigzag topologies have been shown to be useful both for formal variation (Helm et al. 2015; Yuan et al. 2016; Soler et al. 2017; BranchTech 2018) or structural efficiency (Tam et al. 2018). In many among this line of work, the industrial robot follows a manually assigned zigzag end effector path, with limited variation in the end effector’s direction. Recently, Huang et al. (2016) and Yu et al. (2016) presented a constrained graph cut algorithm to tackle the extrusion sequence planning problem. Their algorithm solves for a feasible extrusion sequence along with end effector’s feasible directions for each extrusion step, which enables fabrication of frames with nonrepetitive topologies. However, their method abstracts away the robot’s kinematics during the sequence planning process and instead uses an ad hoc method to generate feasible guiding curves for the robot’s end effector to follow. This results in slow computation and lacks any trajectory feasibility guarantee. As a step forward from their work, this work formulates the problem of combined sequence and motion planning for the first time and proposes an algorithm to solve it in an integrated, streamlined way.
2.2 Path planning for architectural robotic assembly
Robotic spatial extrusion, although regarded as a fabrication instance, shares a similar planning problem formulation with a broader class of robotic assembly problems: given a discrete spatial structure’s design model, the robot needs to be assigned a coordinated sequence of transition and assembly actions, to manipulate or fabricate individual elements in a specific order to construct the designated design. The solutions for these problems are alternating sequences of transition and assembly actions that corresponds the robot moving with its hand empty and while holding or extruding the element to be constructed. The assembly action usually involves constrained Cartesian movement of the robot’s end effector, while transition involves freespace movement with no extra constraint other than collisionfree. Specifically for robotic spatial extrusion, the assembly action corresponds to the extrusion process.
Existing exploration of robotic assembly in different architecturescale application contexts has focused on the design of applicationspecific processes and associated hardware systems (Gramazio et al. 2014). In all these applications, researchers have encountered similar problems: (1) the assignment of an order to assemble objects and (2) the generation of feasible robotic trajectories that do not collide with objects in the workspace (Parascho et al. 2017; Eversmann et al. 2017; Søndergaard et al. 2016). Current solutions to this problem typically involve an intuitionbased trialanderror method. For a given robot configuration during assembly, designers manually specify endeffector poses on the assembly geometry to achieve linear endeffector movement. For transition trajectories, designers manually generate guiding curves for the endeffector to follow, which hover over the workspace within a safety distance. Utilizing industrial robot’s builtin commands like Linear movement (LINE) or PointToPoint (PTP) (Braumann and BrellCokcan 2011), users rely on the builtin interpolation method to translate endeffector assignment to joint trajectories that are free of collisions, respect joint limits and avoid singularities. As a result, this requires significant effort to diagnose the planning failure in a trialanderror manner. Software packages exist to support this trialanderror procedure by simulating robotic motion [such as HAL (Schwartz 2012) and KUKAPRC (Braumann and BrellCokcan 2011)], but these tools can only simulate/test a robotic trajectory based on Tool Center Point (TCP) planes and joint configurations input, without the ability to automatically plan a collision and kinematicsaware trajectory. Because of this, these tools currently support a suboptimal manual planning process.
Recent attempts at addressing this problem include partial solutions: (1) use an automated sequence planning scheme and then use either the manual planning process described above or some ad hoc trajectory finding technique without trajectory feasibility guarantee (Søndergaard et al. 2016; Yu et al. 2016; Huang et al. 2016) (2) use a manually assigned assembly sequence and then use motion planning/online control algorithms to find feasible trajectories (Gandia et al. 2018; Dörfler et al. 2016). While these approaches might be feasible for models with simple and sparse topologies, the construction sequence and robotic motion is much more nuanced for designs with denser material distribution and nonstandard topologies. An integrated planning tool that combines assembly sequence and motion planning has not been developed. As pointed out in Giftthaler et al. (2017), the combination of an autonomous control scheme with a “higherlevel planner” that is “able to negotiate cluttered environment” is a key step to enable robotic assembly systems to operate safely in densely populated workspaces (Eversmann et al. 2017). This work presents the first rigorous formulation of this sequence and motion planning problem, provides a new algorithm to solve it as a baseline, and shows the major computational overhead emerging in these problems. Thus, this work sets up the baseline for comparison for future development for highlevel sequence and motion planners.
2.3 Spatial extrusion in computer graphics
In recent years, there is a growing interest in the computer graphics research community in exploring new algorithms and hardwares to enable rapid prototyping and fabrication of visual objects (Baudisch and Mueller 2017). In this line of research, the exploration on spatial 3D printing started with the work WirePrint (Mueller et al. 2014), which proposes an efficient way to print wireframe meshes, where edges in the mesh are directly extruded in 3D space. A wireframe of a model is generated by slicing it horizontally and filling each slice with zigzag filaments. This approach is limited in the types of meshes that can be printed. To improve flexibility, Peng et al. (2016) introduce a 5DOF printer that modifies a standard delta 3D printer by adding two rotation axes to the print bed. Following up this work, Wu et al. (2016) present a printing sequence planning algorithm for this 5DOF printer. Peng et al. (2018) propose a new human–machine interaction scheme using augmented reality, enabling a versatile live editing interface to link designer’s modeling and robotic spatial extrusion.
Related problems In the context of 3D printing in bioengineering, Gelber et al. (2018a) presented a heuristic backtrack searching algorithm to generate printing sequence to enable microscale spatial 3D printing on a purposebuilt isomalt 3D printer. They were the first to identify that joint positioning errors are caused by beam compliance and include it as a cantilever constraint in the sequence searching process (Gelber et al. 2018a, b). This finding influenced the nodal printing orders routing part of the sequence planning module presented in this work (Sect. 3.3.2).
2.4 Manipulation planning
The robotic planning community has developed many approaches for motion planning that identify trajectories by searching in the continuous space of robot joint angles. Recent approaches perform this search using either sampling (LaValle 2006) or optimization (Ratliff et al. 2009; Kalakrishnan et al. 2011; Schulman et al. 2014). In manipulation planning, the goal is not only to move robot without colliding with objects in the environment, as in classical motion planning, but also to contact, operate, and interact with objects in the world. Early treatment of this problem uses a manipulation graph to decompose planning for one robot to one object into several problems that each requires moving between connected components of the combined configuration space (Alami et al. 1994; Siméon et al. 2004). This work observes that solutions are alternating sequences of transit and transfer paths, which corresponds the robot moving with its hands empty and while holding an object. Hauser and Latombe (2010) and Hauser and NgThowHing (2011) identify a generalization of manipulation planning problem as multimodal motion planning, i.e., motion planning for systems with multiple modes, representing different submanifolds of the configuration space subject to different constraints.
Rearrangement planning is a special instance of pickandplace planning where all objects have explicit goal poses. These problems are very similar to the robotic extrusion planning problems addressed in this work, where object goal poses are specified in the input design model. Stilman et al. first introduced a version of the rearrangement problems called navigation among movable obstacles (NAMO), where the robot must reach a specified location among a field of movable obstacles (Stilman and Kuffner 2008; Stilman et al. 2007). They provide a greedy backchaining algorithm for solving monotone problem instances, where each object need only be moved onces. Extending this work to nonmonotone problem instances, Krontiris and Bekris (2015, 2016) provided an algorithm that constructs a probabilistic roadmap (PRM) (Kavraki et al. 1996) in the combined configuration space, using the algorithm of Stilman et al. as connection primitive.
Dogar et al. (2015) propose a formulation of multirobot grasp planning as a constraint satisfaction problem (CSP). They attempt to find short plans that requires few regrasps. However, they assume that an assembly sequence is given and does not consider reachability constraint between assembly configurations.
2.5 Task and motion planning
While motion planners deal with geometric constraints in highdimensional configuration spaces, they do not consider abstract features of the domain, i.e., they can plan how to move the robot’s joints to pick up an object but cannot decide the order of tasks to satisfy certain constraints. In contrast, the artificial intelligence (AI) planning community considers problems that are discrete but require many types of actions to be performed over long horizons without predefined horizon length (McDermott et al. 1998; Helmert 2006). Recent work in task and motion planning (TAMP) (Dornhege et al. 2012; Toussaint 2015; Lagriffoul and Andres 2016; Garrett et al. 2018a) combines AI and motion planning to simultaneously plan for discrete objectives as well as robot motions. This work aims to enable robots to operate in applications, such as cooking, which require discrete choices of which objects to grasp or cook as well as continuous choices of which joint angles and object poses can physically perform each task. A key challenge is that often physical constraints such as collision, kinematic, and visibility constraints can restrict which highlevel actions are feasible. Readers are referred to Garrett et al. (2018b) for a more complete review of the work in this area.
Lagriffoul et al. (2014) propose a constraintsatisfaction approach to interleave the symbolic and geometric searches. They focus on limiting the amount of geometric backtracking. LozanoPérez and Kaelbling (2014) take a similar approach but leverage CSPs operating on discretized variable domains to bind free variables. The sequence planning module (Sect. 3.3) proposed in this work adopts a similar technique using CSP to bind free geometric variables on a plan skeleton. However, it relaxes the requirement on feasible whole paths’ existence and trades the algorithm’s completeness for scalability.
2.6 Sequence and motion planning for spatial extrusion: unique challenges and unmet needs
The sequence and motion planning (SAMP) problem in spatial extrusion context is a subclass of highdimensional robot manipulation problems, or more generally, task and motion planning (TAMP) problems, which require planning a coordinated sequence of motions that involve extrusion as well as moving through free space. The SAMP problems involve two key aspects that differ them from typically studied TAMP problems: (1) fixed but long planning horizon, (2) physical constraints that involve computation overhead.
First, the discrete horizon of the SAMP problems is much longer than many TAMP benchmarks (Lagriffoul 2016), which often only require manipulating a couple of objects. Because each element must be extruded once and the goal object poses are specified by the input design geometry, the planning horizon equals to the number of elements in the input model, which is known in advance. Thus, SAMP requires identifying an order for extruding linear elements, fitting this order to a fixedlength plan skeleton, and binding the required geometric parameters. In contrast, TAMP problems generally have unsettled action plans—it is not initially clear which actions are needed and in which order to perform these actions to complete a task and, thus, the planning horizon can be arbitrarily long.
Second, SAMP problems involve physical constraints such as stiffness and stability that are not typically found in TAMP benchmarks. These constraints impact many state variables at once, making them challenging to effectively incorporate in many discrete task planning algorithms. Rather than directly using existing TAMP algorithms, a specialized system is developed in this research that incorporates several existing ideas but, because of its specialization to assembly planning, can scale to complex models.
Apart from the planning problems’ characteristics, common task specification languages for discrete AI planning systems, such as Planning Domain Definition Language (PDDL) (McDermott et al. 1998), are not intuitive for architects and designers. The requirement of specifying task domains, predicates, action’s preconditions and effects departs from the architectural language of shape and geometry and, thus, creates a gap between an architect’s geometric model and robotic task specification for planning. This gap in the modeling interface inhibits these algorithms from being easily adapted to architectural robotic assembly applications.
In summary, there is a rich literature of work related to robotic spatial extrusion for architecture, ranging from theoretical research in robotic task and motion planning to examples of built work of considerable intricacy. However, the field is nevertheless lacking an integrated, generalpurpose method that can be applied systematically to handle the geometric and topological complexity of 3D trusses in the context of contemporary architectural design. This work addresses this gap by presenting a new sequence and motion planning algorithm framework and a modularized implementation. Although the algorithm described in this paper is more specialized than most of the TAMP approaches, the ability to scale to problems with much longer planning horizon and a larger branching factor is one of the key focuses of this research.
3 Sequence and motion planning for robotic extrusion
This section first presents a new problem formulation for the sequence and motion planning (SAMP) problem involved in robotic extrusion (Sect. 3.1). Then, a new algorithmic framework is presented to solve the SAMP problem. Section 3.2 gives a conceptual overview of the framework’s hierarchy and introduces its three main modules. Detailed descriptions are then stated for the sequence planning module (Sect. 3.3), the motion planning module (Sect. 3.4) and the postprocessing module (Sect. 3.5).
Assumptions In this paper, the robot is assumed to work in a fully observable and deterministic environment. The generated plan is purely geometric—the computed velocities are not used in the execution. Trajectory’s speed for execution is reassigned separately by the user after the planning is finished and robot’s position control is carried out by the industrial robot’s controller.
Model input The extrusion planning problem starts with a 3D truss model from a designer. The 3D truss is represented via a standard nodemember data representation. Nodes are described with 3D spatial coordinates in an indexed list. Linear members are described by their start and end node indices. A uniform cross section and material property are assumed for all the elements. Each linear element specifies a sequence of path points that the end effector’s tip must extrude along.
3.1 Planning problem formulation
The extrusion sequence planning problem has a predefined plan skeleton, or action sequence, that has an alternating pattern on the actions: extrude(element \(O_i\))transit − extrude(element \(O_{i+1}\))transit. The planner needs to assign a correct assignment of object \(O_i\) to each action in the plan skeleton and bind variables to fully specify robot’s configurations during and between extrusion steps, where different constraints exist for the robot’s motion in the two types of actions. In the following discussions, let n denote the total number of elements to be extruded in the model.
3.1.1 Freespace joint motion and semiconstrained Cartesian motion
The robot’s motion is subject to different constraints during transition and extrusion process in the action sequence. The transition process involves a segment of the robot’s trajectory after it finishes extruding the last element, and before it starts extruding the next one. Thus, this connecting trajectory requires only a collisionfree joint trajectory with no extra requirement on the position and the orientation of the end effector, i.e., freespace joint motion.
3.1.2 Constraint variables

\(O_i, \,i \in \{1, \dots , n\}\): The element assignment for the ith extrusion action. Its value domain is \(\{1, \dots , n\}\), representing the indices of elements in the input model.

\(H_i, \,i \in \{1, \dots , n\}\): The end effector’s orientation for the ith extrusion action. As described in Sect. 3.1.1, the domain of \(H_i\) can be discretely parameterized by a direction vector \(\mathbf {v}\) on a unit sphere and a rotation angle \(r \in [0, 2\pi )\).

\(K_{i,j}, \,i \in \{1, \dots , n\}, \,j \in \{1, \dots , E_i^d\}\): the robot’s configuration with its end effector extruding at ith extrusion element’s jth path point. The domain of \(K_{i,j}\) is a set of joint values of a robot. \(E_i^d\) denotes the path point discretization number of element i.
Constraints Constraints relate the variables to one another and limit the set of valid assignments. If all the constraints are collectively satisfiable, then a plan skeleton is valid, and the pruned geometric variable domains specify the geometric details for EE’s poses during extrusion. The CSP formulation includes the following constraints:
AllDiff(\(O_1, \dots , O_n\)): Each element is used only once by an extrusion action. Thus, all extrusion element assignment \(O_i\)’s values are distinct. This constraint is equivalent to enforce that the resulting assignment \(\{O_i\}\) is a permutation of \(\{1, \dots , n\}\).
Stiffness(\(O_1, \dots , O_k\)), \(k = 1, \dots , n\): The stiffness constraint ensures that the partially extruded structure is stiff at each assembly step and the maximal deformation due to gravity (or other constantly presented load) is bounded by a predefined tolerance. The deformation of all the nodes under gravity can be calculated using finite element analysis of linear frame structures (McGuire et al. 1999). The constraint test function returns true if and only if the maximal node deformation is smaller than a tolerance.
Stability(\(O_1, \dots , O_k\)), \(k = 1, \dots , n\): The stability constraint checker returns true if the gravitational center’s projection on the supporting plane lies in the convex hull of all the grounded nodes and returns false otherwise. It guarantees that the rigid, partially extruded structure meets moment equilibrium and does not require a tension connection at the support to remain upright (Phear 1850). This calculation can be easily integrated with the stiffness constraint’s computation, by evaluating if there exists a tensile force among the reaction forces in the grounded elements.
ExistValidFreeSpaceMotion(\(O_{\text {static}}, O_1, \dots , O_k, K_{O_{k1}, E_{O_{k1}}^d}, K_{O_k, 1}\)), \(k = 2, \dots , n\): Viewing already extruded elements \(O_1, \dots , O_{k1}\), together with static scene objects \(O_{\text {static}}\), as collision objects, this constraint checks if there exists a free space joint motion to connect from last robot’s configuration \(K_{O_{k1}, E_{O_{k1}}^d}\) for extruding element \(O_{k1}\) and the first robot’s configuration \(K_{O_k, 1}\) for extruding the next element \(O_{k}\).
ExistValidExtrusionMotion(\(O_{\text {static}}, O_1, \dots , O_k, H_{O_k}, K_{O_k,1}, \dots , K_{O_k,E_{O_k}^d}\)), \(k = 1, \dots , n\): This constraint tests if there exists a collisionfree extrusion motion to have the robot’s EE linearly traverse the path points \(pt_{j}(O_k), j=1,\dots ,E_{O_k}^d\) defined by the element \(O_k\) with EE orientation \(H_k\) (Sect. 3.1.1). The compatibility between the robot’s configurations and the EE poses is enforced by checking InverseKinematics (\(H_{O_k}@pt_{j}(O_k)) = K_{O_k,j},\,j=1,\dots ,E_{O_k}^d\). Elements \(O_1, \dots , O_{k1}\) and static objects \(O_{\text {static}}\) are considered as collision objects and checked against robot’s configurations.
Notice that there are three main features that make the CSP problem here hard to solve: (1) there are a large number of robot configuration variable \(K_{i,j}\) to be binded. (2) The geometric constraint variable, \(H_i\) and \(K_{i,j}\), requires a discretization of the end effector pose space and the robot’s joint space. The discretization granularity is a metaparameter that is essential to balance the computation’s completeness and tractability. (3) The Stiffness and Stability involve physical simulation, while ExistValidFreeSpaceMotion and ExistValidExtrusionMotion involve calling singlequery motion planner or kinematics solver. The evaluation of all these constraints commonly induces a large amount of overhead as they will be called many times by the CSP planner and they impact many variables at the same time.
To get around this computational complexity, a specialized hierarchical planning algorithm is proposed to break this problem into isolated, simplified subproblems. The algorithm trades its completeness for tractability, especially to harness the long planning horizon induced by complex design inputs.
3.2 Conceptual overview of the planning framework
These modules, along with the framework inputs and outputs, are described in greater detail in the following sections. An example problem of using a fixedbase 6axis robot is used to illustrate the details of each module, but the system is general and can also apply to other robotic extrusion systems.
3.3 Sequence planning module
The proposed sequence planner takes a 3D truss model as input and solves for the order of the extrusion operation and associated feasible endeffector directions. The sequence planner solves a relaxed version of the original CSP problem specified in Sect. 3.1 with a standard backtracking search strategy. The backtracking search is guided by an applicationspecific variable ordering, forward checking, and constraint propagation. The novel contribution of this sequence planning module is: (1) the relaxed CSP formulation; (2) some empirical computation statistics from a straightforward guided backtracking search solver, to demonstrate the computational bottleneck and establish the first baseline for future algorithmic development and comparison.
3.3.1 Relaxed CSP constraints
In the original CSP encoding in Sect. 3.1, the robot configuration variables \(K_{i,j}\) tie the two constraint ExistValidFreeSpaceMotion and ExistValidExtrusionMotion together, inducing large computational overhead. To avoid this, a relaxed CSP formulation is proposed to eliminate the robot configuration variables and simplify the motion constraints.

\(O_i, \,i \in \{1, \dots , n\}\): The element assignment for the ith extrusion action as before.

\(V_{i,j}, \,i \in \{1, \dots , n\}, \,j \in \{1, \dots , m_{V}\}\): The end effector direction’s feasibility for the ith extrusion action. The index j refers to an ordered list of unit vectors uniformly sampled on a unit sphere (Fig. 4a). The size of this sampled vector list is \(m_{V}\). The value domain of \(V_{i,j}\) is \(\{0, 1\}\), which represents the direction is feasible (0) or not (1).
Constraints The constraints AllDiff, Connect, Stiffness and Stability are kept in the relaxed CSP formulation. The motionrelated constraints ExistValidFreeSpaceMotion and ExistValidExtrusionMotion are replaced with constraints on EE directions, which are simplified and easiertocheck:
ExistValidExtrusionEEPose(\(O_{\text {static}}, O_1, \dots , O_k, V_{O_k,1}, \dots , V_{O_k,m_V}\)), \(k = 1, \dots , n\): This constraint checks if there exists one valid end effector direction for extruding element \(O_k\). Existing elements \(O_1, \dots , O_{k1}\) and \(O_{\text {static}}\) are considered as collision objects. These collision objects may collide with the end effector if extruding with some specific EE directions (Fig. 4b).
The subfunction ExistValidKinematics checks if there exists valid robot kinematic solutions to extrude along element \(O_k\)’s path points with EE direction \(V_{a}\). The constraint checker samples a rotation angle \(r \in [0, 2\pi )\), which fully determines a list of EE poses during the extrusion of element \(O_k\) when combined with direction \(V_{a}\) and path points \(pt_i(O_k)\). The subfunction checker returns true if there exists a collisionfree inverse kinematics solution for each one of the EE poses and returns false if no solution is found within a userspecified timeout.
3.3.2 Solving the CSP
A key advantage of a CSP formulation is that if a user provides a description of their problem in this representation, a generic solver can perform the search. In this paper, a simple backtracking search with value ordering and constraint propagation is used as a baseline solver. A domaindependent heuristic is proposed to assist the value ordering. In addition, to limit the computation in a reasonable amount of time, a userguided model decomposition is introduced before running the search algorithm. A winning strategy of only incorporating constraint propagation and model decomposition is selected based on numerical runtime results.
While integrating the CSP encoding with generic, blackbox CSP solvers is left as future work, the key contribution of this section is to establish a baseline for future algorithmic development to compare against. By providing detailed computation statistics, computation bottleneck on checking physical and geometrical constraints can be observed that distinguishes extrusion planning problems from general discrete CSP problems.

The algorithm selects unassigned variable by simply advancing along the time index k. It only searches on the extrusion object assignment variable \(O_k\) and leaves EE direction variables \(V_{i,j}\) pruned by constraint propagation. Thus, the maximal depth of the search tree is n.

A domaindependent heuristic is used to guide the value ordering (line 7, Algs. 1).

The TestConsistency function (line 8, Algs. 1) involves checking physical and geometrical computation, which might creates computational overhead.

The constraint propagation (line 10, Algs. 1) assumes the existence of element \(O_k\) and updates the EE direction \(V_{O_i,j}\)’s domain for all the unassigned elements \(O_i\). This invokes a large amount of collision checking between the element \(O_k\) and the EE extruding along future element \(O_i\) with directions that are still in \(V_{O_i,j}\)’s feasible domain.
Backtracking search algorithm runtime statistics
Model  E  Decomp.  Collision cost  Total time (s)  Partial states visited  Test stiffness and stability [time (s)cnt]  Test kinematics [time (s)cnt]  Update EE direction state [time (s)cnt]  Computing collision cost [time (s)cnt]  

Fig. 5a  23  \(\times\)  \(\times\)  9.8  39  0.72  261  6.38  273  2.7  39  –  – 
\(\checkmark\)  18.94  23  0.36  150  0.94  160  2.64  23  14.78  160  
Fig. 5b  84  \(\times\)  \(\times\)  \(>6\) h  –  –  –  –  –  –  –  –  – 
\(\checkmark\)  \(\times\)  64.43  90  5.67  481  17.40  621  41.11  90  –  –  
\(\times\)  \(\checkmark\)  580.35  84  8.93  1402  9.87  1538  30.65  84  528.19  1528  
\(\checkmark\)  \(\checkmark\)  79.34  84  4.01  446  4.21  582  30.52  84  40.40  581  
Fig. 5c  132  \(\times\)  \(\times\)  \(>6\) h  –  –  –  –  –  –  –  –  – 
\(\checkmark\)  \(\times\)  103.82  142  9.54  715  14.74  928  79.35  142  –  –  
\(\times\)  \(\checkmark\)  1871.38  136  34.49  3099  31.63  3313  77.04  136  1710  3262  
\(\checkmark\)  \(\checkmark\)  147.42  132  9.80  704  8.10  914  75.34  132  53.54  890 
Userguided model decomposition Model decomposition involves grouping the discrete input model into several connected components. Taking advantage of a user’s intuition on the geometric relationship, the decomposition breaks the whole sequencing problem into several smaller ones and the search is confined to each of these small subproblems. This decreases the size of the search space and leads to more efficient CSP solving overall. When the input model has a large number of elements, the runtime caused by computing the collision cost increases drastically, as it is equivalent to call UpdateEEDirectionState everytime a cost is computed. The model decomposition limits the states considered by the constraint propagation.
Backtracking search computation statistics A computation runtime study is presented in Table 1 to compare the impact of the heuristic value ordering, model decomposition, and consistency checking on runtime. The backtracking search algorithm is applied to three models shown in Fig. 5, with some algorithmic features turned on/off.
From the total runtime shown in Table 1, a vanilla backtracking strategy without model decomposition and variable ordering quickly becomes incapable with the increase of element number in the model. With the collision cost turned on but without decomposition, the search algorithm is capable of finding a solution but spends nearly 90% of its time computing the collision cost. This overhead is caused by the massive collision checking for doing constraint propagation on a large number of variables’ domain. With the decomposition turned on, the runtime for computing collision cost quickly drops 97%, because the decomposition limits the variables’ domain that needs to be propagated upon to the ones within the local decomposition group. In addition, the usage of collision cost reduces the number of backtracking and the runtime for testing kinematics, as the collision cost helps prioritizing elements with less chance to cause infeasible kinematics solutions. But this reduction is not sufficient to cover the runtime increase caused by the collision cost’s computation.
In contrast, the search algorithm performs the best with the decomposition but without collision cost in all the three models presented. The winning strategy of the proposed backtracking search algorithm would be using the model decomposition without using the collision cost to dynamically order the values.
The model (a) (Fig. 5a) can be viewed as a single layer in a bigger model because of its small number of elements. Even without any backtracking (Fig. 5a—without collision cost, Table 1), the search algorithm spends 7% of its time checking stiffness and stability constraint, 65% checking kinematics, 28% updating EE direction states. This simple case shows that the computation overhead is induced by the geometric and physical constraints. Thus, the extrusion planning problem is dominated by geometrical and physical aspect of the problem, rather than the logical aspect of it. This feature distinguishes this specific class of CSP problem from the common discrete CSP problems considered in the classic AI community and, therefore, deserves special algorithmic treatment.
Remarks: Notice that both updating EE direction states and computing collision cost involve a large amount of twobody collision checking between an element and the end effector extruding along another element. In this work, this twobody collision checker is implemented using an ad hoc and handwritten lineface intersection to approximate the elementEE collision. In future work, this collision checker can be replaced a more modern collision checker that uses advanced algorithms, e.g., FCL library (FCL 2018) with the GJK algorithm (Gilbert et al. 1988) implemented. The authors expect at least 30% of speedup in updating EE direction state and computing collision cost if collision checker is replaced.
Routing nodal printing orders After the CSP planner finishes its search and produces an extrusion order, the nodal printing orders can be further optimized to increase empirical printing success. For the steps that connect two existing nodes, the assignment of start node and end node can be chosen without affecting the sequence planning result’s feasibility. This assignment has recently proven to be critical for the physical execution of spatial extrusion due to the molten joint’s incapability to resist bending moment and elastic recoil effect (Gelber et al. 2018b). Gelber et al. (2018a) introduce a cantilever constraint to their extrusion planning algorithm to address this problem: new elements cannot be connected to node p, if any previously printed element connected to p is cantilevered. A relaxed version of this constraint is used here to route the nodal printing order: starting from the node with a larger degree (number of connected elements) is preferred. Based on the authors’ experiments, the introduction of this direction routing process dramatically increases the rate of empirical printing success while not affecting the geometric planning.
3.4 Motion planning module
The plan skeleton obtained from the sequence planner specifies the order \(O_1, O_2, \dots , O_n\) and a range of collisionfree endeffector directions \({\mathbf {v}_{O_i,j}}, \, i \in \{1,\dots ,n\}, \, j \in \{1,\dots ,m_i\}\). Notice that this list of vectors corresponds to the directional vectors with boolean constraint variable \(V_{O_i,j'} = 0,\, j' \in \{1,\dots ,m_V\}\). Their \(j'\) indices are renumbered to \(j \in \{1,\dots ,m_i\}\), where \(m_i\) is the total number of collisionfree directions for extruding element \(O_i\). In this section’s description, the vectorial variables are written in bold notation.
To obtain a full motion for the robot, the motion planner needs to (1) determine the robot’s trajectory during each extrusion and (2) plan for the robot’s trajectory between adjacent extrusions. This is a dual motion planning problem due to the Cartesian motion planning with constraints on endeffector’s poses during extrusion task and free motion planning without constraints on end effector’s pose in transition.
In this work, this dual motion planning problem is solved in two phases: semiconstrained Cartesian planning (Sect. 3.4.1) to resolve the redundancy in end effector poses and associated robot kinematics during each extrusion task. Then, retraction planning (Sect. 3.4.2) is added between the Cartesian motion and transition motion to enable a safer robot trajectory. Finally, transition planning (Sect. 3.4.3) is used to compute robot’s trajectory in between adjacent extrusion tasks. The sequential layout of transition motion, retraction motion, and Cartesian motion is illustrated in Fig. 6.
3.4.1 Semiconstrained Cartesian planning
In many robotic assembly applications, the robot’s end effector is required to move linearly, where the end effector’s tip must follow designated path points. However, its orientation may still have some degrees of freedom (De Maeyer et al. 2017). In the case of spatial extrusion, the tip of the printing nozzle needs to traverse the points on the linear path formed by the element but has freedom in choosing the end effector’s orientations. In addition, even when the endeffector’s poses are fully specified, there is still kinematic redundancy in choosing corresponding robot configurations. The planning for this type of motion is called semiconstrained Cartesian planning.
Notice that the inverse kinematics solution \(\mathbf {J}_{k_i}\) for target endeffector pose is not unique and needs to be determined by the planning algorithm. Meanwhile, the computed joint solutions have to be collision free with respect to objects in the environment during the extrusion. In addition, the motion between consecutive joint solutions should respect the robot’s maximum velocity and acceleration limitations so that the joint solution sequence is physically executable.
Existing work addresses semiconstrained Cartesian planning problem using an approach that discretizes the end effector’s candidate poses and kinematic solutions and performs a discrete search on a planning graph (De Maeyer et al. 2017; ROSI 2018a). This algorithm starts with a list of given end effector poses for the robot to traverse and each end effector pose is assigned with parameters with tolerance ranges. With the tolerance, each given path pose represents a family of parameterized end effector poses and each pose in this family corresponds to a family of robot’s joint configurations by performing analytical inverse kinematics with ikfast (IKFast 2018). These joint configurations can be organized as vertices in a planning graph where edges only exist between vertices that belongs to the same or adjacent end effector pose families. Vertices that represent joint configurations in collision are pruned and edges that represent sharp turns of adjacent joint configurations will not be added to the planning graph. A cost is assigned to each edge in the graph as the \(L_1\) norm of the difference of the two adjacent joint configurations. In this way, the semiconstrained Cartesian path planning problem is converted to a shortest path searching problem on a directed ladder graph, which is a multipartite graph with edge connections between only independent set k and \(k+1\), \(k \in \{1, \dots , n1\}\). Each rung in the ladder graph consists of joint configurations that belong to the same end effector pose’s parameterized family. The rung’s index can be viewed as a time index on path points. The resulting path represents a sequence of joint configurations with minimal joint difference between adjacent joint configurations (De Maeyer et al. 2017).
Extracting sparse ladder graph This section first introduces a sparse representation of the planning graph, called sparse ladder graph, to help compress and locate the region on the planning graph that contains a closetooptimal solution for the semiconstrained Cartesian planning problem. A samplingbased planning algorithm is used to extract this sparse representation that is asymptotically optimal locally in this module, which means that the probability of converging asymptotically to the optimum approaches 1.00 with an infinite number of samples (Karaman and Frazzoli 2011).
There are two reasons for the memory overhead in the original planning ladder graph: (1) the entire ladder graph needs to be expanded and stored and (2) time indices assigned to workspace path points lead to a massive number of edges connecting rungs that have adjacent time indices. This observation leads to the idea of leveraging the extrusion sequence \(\{1,\dots , n\}\) as a sparser time index for rungs and incrementally building a sparse ladder graph to first find end effector poses for each assembly task and later recover a reduced ladder graph to search for joint configurations. A sparse ladder graph is a compressed version of the original ladder graph, where joint configurations are grouped under a compact data structure called capsule and directed edges are constructed between capsules.
A capsule is a data structure that contains an extrusion element’s index i, an EE direction \(\mathbf {v}\), an EE rotation angle r, and two sets of feasible robot joint configurations for the first and last EE poses specified by the element’s path points and orientation \((\mathbf {v}, r)\). The feasible robot joint configuration sets for the first and last extrusion EE pose are denoted by \(\{\mathbf {J}_1\}\) and \(\{\mathbf {J}_{1}\}\), respectively. In a word, a capsule is a fivetuple \((i, \mathbf {v}, r, \{\mathbf {J}_1\}, \{\mathbf {J}_{1}\})\) that stores the information to describe EE poses for an element’s extrusion and the robot’s feasible joint configurations at the start and the end of the extrusion. A graphical demonstration of capsule’s definition is shown in Fig. 7.
This definition of capsules enables (1) the definition of cost (or distance) on a directed edge between two capsules based on first and last pose’s joint configurations and (2) the later expansion to a full planning graph of joint configurations. Graph edges in the sparse ladder graph are directed and limited to capsules that have adjacent time index, i.e., between \((i, {\mathbf {v}}_i, r_i, \{{\mathbf {J}}_1\}_i, \{{\mathbf {J}}_{1}\}_i)\) and \((i+1, \mathbf {v}_{i+1}, r_{i+1}, \{\mathbf {J}_1\}_{i+1}, \{\mathbf {J}_{1}\}_{i+1})\). The cost of such an edge is defined as the minimal \(L_1\) norm of joint pose difference between the joint configurations in \(\{\mathbf {J}_{1}\}_i\) and the ones in \(\{\mathbf {J}_{1}\}_{i+1}\). By searching on the sparse ladder graph, one can locate closetooptimal end effector poses for all the extrusion tasks, without encountering the memory overhead caused the expansion of joint configurations for all the path points.
Notice it is still possible that one can experience memory overflow problem with this sparse ladder graph approach if considering a model with extremely large number of elements. However, the authors find that this sparse ladder graph approach keeps a low memory occupation for all the experiments that have been carried out and thus the approach is considered sufficient for most of the 3D trusses in the architectural design context.
Computing an optimal capsule path on the sparse ladder graph The sparse ladder graph is used to find a path of capsules to traverse all the extrusion tasks that is closetooptimal locally in this module. This capsule path can be expanded to a fraction of the original planning graph that contains the closetooptimal path of joint configurations. Samplingbased algorithms are well suited for this problem because they allow an incremental construction of the sparse graph and provide almostsure convergence to the optimal solutions locally for this module (Karaman and Frazzoli 2011).
To apply these algorithms to the problem here, special initialization, sampling, feasibility checking, and connecting functions are provided. These procedures adapt planning to the hybrid discretecontinuous state space and the sequential layout of the sparse ladder graph.
Let \(X = ([m_1] \times [0, 2\pi )) \times ([m_2] \times [0, 2\pi )) \times \dots \times ([m_n] \times [0, 2\pi ))\) be the full state space of the sparse planning problem, where \([m_i] \times [0, 2\pi )\) parameterizes the endeffector’s pose by assigning EE direction with index \(j \in [m_i] = \{1, \dots , m_i\}\) in a precomputed list of directions and rotation angle \(\theta _i \in [0, 2\pi )\). n is the number of elements to be extruded and represents the time index in the extrusion process. For a specific extrusion task i, one can consider the taskprojected state space \(X_i = [m_i] \times [0, 2\pi )\).
A state \(x_i\) in the projected state space \(X_i\) corresponds to a capsule \((i, \mathbf {v}, r, \{\mathbf {J}_1\}, \{\mathbf {J}_{1}\})\). Notice that \(\{\mathbf {J}_1\}, \{\mathbf {J}_{1}\}\) are not independent variables, but dependent on \((i, \mathbf {v}, r)\).
In this paper, the Rapidly exploring Randomize Tree* (RRT*) algorithm is applied to the sparse ladder graph (Fig. 8). Other asymptotically optimal samplingbased algorithms, e.g., Probablistic Roadmap* (PRM*), could also be used. The complete description of these algorithms can be found in Karaman and Frazzoli (2011). Key modifications enabling these algorithms to operate on the sparse ladder graph are highlighted below:
Sample: The sampler operates on a hybrid discretecontinuous state space, which returns a state \(x_i \in X_i\) that is generated from three different and standalone samplers. Each one of these three samplers generates independent samples by randomly sampling uniformly the corresponding variable domain. The generated samples uniquely determine (1) extrusion task’s time index \(i \in [n]\), (2) endeffector direction index \(j \in [m_i]\) in task i’s feasible EE direction list and (3) rotation angle \(r \in [0, 2\pi )\), which all together determine a state \(x_i = (\mathbf {v}_j, r)\).
CheckFeasibility: A state \(x_i\)’s validity is verified by checking if all the encoded path poses have feasible robot kinematics solutions. A kinematic solution for a given end effector pose is pruned if it results in a collision. Each task has a different set of collision objects, as elements extruded in previous tasks become collision objects in subsequent tasks.
Nearest and Rewiring: Given a state \(x_i \in X_i, 1 < i \le n\), the function Nearest returns the vertex \(x_{i1} \in G \cap X_{i1}\) with the smallest cost to \(x_i\), where G is the current sparse ladder graph. Edge connections are restricted to only vertices in adjacent extrusion tasks, as skipping tasks is not allowed.
Extracting a trajectory solution The samplingbased algorithm returns a path \(\delta\) in the sparse ladder graph. The path is then expanded as a subgraph of the original planning graph to enable the use of standard shortest path search algorithms to find the sequence of joint poses with minimal cost. Each state \(x_i\) in the returned path \(\delta\) is expanded by adding the intermediate path points’ kinematic solutions as vertices on the corresponding rungs and then constructing edges between all vertices on adjacent rungs, which corresponds to two successive path points (Fig. 8).
The expanded graph is a directed acyclic graph (DAG). By topologically sorting its vertices, a shortest path can be identified in time linear in the size of the graph (chapter 24.2, Cormen 2009). The resulting path gives a discretized joint trajectory for each extrusion task in the action sequence that fully determines the robot’s configurations during extrusion.
Notice that when applied to semiconstrained Cartesian planning problems, this sparse graph hierarchical approach preserves local optimality in this module asymptotically, compared to directly applying shortest path search on a full ladder graph of joint configurations. In the original ladder graph, edge connections are limited between joint configurations that belong to the same or adjacent end effector pose parametrization to satisfy the end effector’s orientation constraint. Viewed in the sparse ladder graph, this disallowance of edge connection across pose families is enforced by putting all the capsules that correspond to the same element’s extrusion in the same rung (independent set). Thus, no potential decrease in path is lost by applying this hierarchical sparse ladder graph approach locally in this module. However, if viewed globally on the entire extrusion planning system, the joint configurations generated by this module might result in suboptimal or even infeasible transition trajectories. In general, trading the entire system’s completeness and global optimality for tractability is common among hierarchical planning approaches.
3.4.2 Retraction planning
3.4.3 Transition planning
Following semiconstrained Cartesian planning and retraction planning in the last two sections, transition planning computes a collisionfree joint trajectory connecting the last joint pose in the departing retraction motion in extrusion task i and the first joint pose in the approaching retraction motion in extrusion task \(i+1\). This is solved using a standard singlequery motion planner, which takes into account of the present collision objects in extrusion task i (Fig. 9). The transition planner first tries to call the motion planner for directly connecting the target start and goal configurations. Upon failure, it replans by inserting a reset home waypoint between the start and goal configurations. This guides the planner to find a feasible solution as the configuration space near the home waypoint is less constrained. The transition trajectories generated from three stateoftheart motion planners are shown in Fig. 9. The result in Fig. 9b shows that the CHOMP planner (Ratliff et al. 2009) frequently fails to generate a feasible transition plan on its first attempt and thus requires resetting itself to the home waypoint quite often. Based on the authors’ experience, the STOMP planner works the best, producing smooth and feasible trajectories with less excessive joint movement.
3.5 Postprocessing module
The reassignment of control velocities and synthesis of IO commands The generated robot trajectories is entirely geometrical and its inherent timestamp information only indicates the order of joint configurations. Meaningful timestamps need to be reassigned by the users to the computed trajectories after the planning is finished. In addition, in order to generate instructions for the robot to interact with the physical world, the users need to weave IO commands to synthesize the robot’s motion and end effector’s behavior. Many existing architectural robotic assembly projects involve an offline programming process. In these projects, the insertion of IO commands is usually carried out in a graphical programming platform, for example, Grasshopper (2018), to have a visually friendly way to insert IO commands in the trajectory command list with live simulation playback. This process, however, can be very tedious when working with robotic assembly applications with a long planning horizon and a massive number of configurations. To increase the computed trajectory’s compatibility to these visual programming platforms for trajectory postprocessing, the generated trajectory is formatted in a customized JSON format, which contains a hierarchical information structure to maximize its readability and usability. Each element’s extrusion process contains several subprocesses, each of which is tagged with a subprocess type: transition, retractionapproach, extrusion, or retractiondepart (Fig. 10).
Many robotic assembly projects require the robot to have different end effector speed (also called workspace speed) in different phases of its motion. Users need to produce control velocity subject to the constraint or need of their applications. In the specific case of spatial extrusion, the robot must to extrude material with its end effector following a straight linear movement in a constant speed. Most of the industrial robots provide linear movement commands that take a tool center point (TCP) plane to generate linear movement with a userdefined constant end effector speed. This requires that the result produced by the tagging system contains both robot’s joint trajectory and the associated TCP planes to allow users to choose according to the subprocess’s definition, i.e., “switching modes”. To support this feature, when exporting computed trajectories, the planning system performs forward kinematics to every joint configuration to compute corresponding TCP planes. Both these joint array and TCP array are packed with assembly task id, subprocess id, and subprocess type. In addition, the data type can be specified to indicate what kind of motion the subprocess is using. TCP data should be used if end effector linear movement with constant speed is desired, and joint data should be used if there is no constraint on the end effector’s speed.
On the other hand, control commands for the end effector need to be synthesized into the robotic trajectories. These commands are usually application or hardwarespecific, which involves digital IO, analog IO, and wait (idle) times, to enable industrial robot’s controller to send commands to activate/stop external customized devices’ behavior. For example, spatial extrusion needs the end effector to start extruding material between retraction approach and extrusion motion, and stop extruding right after the robot finishes the extrusion. This is done by inserting a digital ON/OFF command between the designated processes.
To form a smooth transition into the established method of weaving IO commands in a graphical programming environment, the formatted trajectories produced by the postprocessor can be imported into any such platform, such as Grasshopper (2018), with a simple customized parser, to decode the JSON file. Users can insert robot commands, such as digital IO, analog IO, and wait time, based on the assembly element’s index and process context, without having to find the index of a specific joint configuration themselves. Then, existing robot simulation packages can be used to simulate the robot’s trajectory to verify the correctness and safety of the trajectories and export brandspecific robot instruction code.
Applicationoriented path modification For many robot assembly processes, especially spatial extrusion, the variety of end effector designs and material properties requires the incorporation of ad hoc fabrication logic to achieve the desired visual results (Hack and Lauer 2014; Helm et al. 2015) or increase the product’s structural performance (Tam et al. 2018). These fabrication logics, which are derived from physical extrusion experiments, usually involve local modification of an end effector’s pose, such as pressing or extruding following small circular movements at structural joints to create local “knots”.
The metadata associated with the computed trajectories allows users to easily insert these micro path modifications. These path modifications usually require users to iterate on the parameters controlling robot and its end effector’s behavior, until they find the best parameter setting based on experimental observations. For spatial extrusion, one needs to perform many experiments to find the delicate balance between the robot end effector’s moving speed while extruding, cooling air’s pressure, and extrusion rate. Because of the tagging system, the fabrication parameter calibration process repeats between the fine tuning programming platform and physical tests, while keeping the overall planned robotic trajectory unchanged.
4 Implementation
The proposed hierarchical sequence and motion planning framework has been implemented in a proofofconcept planning tool called Choreo. This tool allows users to compute feasible robotic extrusion trajectories using unconstrained target 3D truss geometry, and it supports customized hardware and work environments. This section first presents the general system architecture (Sect. 4.1) and then presents an overview of the user experience of Choreo along each of its computation state (Sect. 4.2– 4.5).
4.1 System architecture overview
4.2 Extrusion problem setup
Robots and end effectors are specified using a Unified Robot Description Format (URDF) file^{2} in Choreo, which is an XML format data that contains robot’s link geometry, joint limitation, and other related data. To specify customized end effector, users need to have the STL mesh for the end effector (used for collision checking) and create a URDF file to link the imported geometry to a desired robot link. Static collision objects in the work environment are imported as STL meshes and linked to the robot’s URDF file.
The geometry of the 3D trusses can be specified using the nodeconnectivity format. A decomposition can be added to the geometry model by simply assigning a layer index to each element. The authors develop a simple parser based on the graphical programming platform Grasshopper (Grasshopper 2018), to have a visually friendly layer tagging workflow. The relative position between the build platform and the robot’s base can be calibrated from the robot and input into the system by a 3D vector using the GUI widget.
4.3 Sequence planning
Currently, Choreo’s sequence planner is powered by a customized backtracking search engine (Sect. 3.3.2). The analytical kinematics computation is performed through the ikfast kinematics plugin (IKFast 2018). The collision check between robot and the environment is implemented using the collision checking interface provided by Moveit! (Sucan and Chitta 2018).
4.4 Motion planning
The semiconstrained Cartesian planner is implemented based on the Descartes planning package from ROSIndustrial (2018a). The sparse ladder graph and the RRT* algorithm is implemented by the author using the Descartes package’s ladder graph data structure. The retraction planner is a direct application of the Descartes package with direction vector sampling.
The transition planner utilizes the motion planner plugin interface of the Moveit! motion planning framework (Sucan and Chitta 2018). Choreo uses the STOMP planner from ROSindustrial (Kalakrishnan et al. 2011; ROSI 2018c) as the main singlequery motion planner, but can be easily configured to work other motion planners.
4.5 Postprocessing and execution
After the motion planning is finished, the computed trajectories are tagged with metadata associated with the extrusion tasks and can be exported as a JSON file. The core module coordinates with the simulation module to display the chosen extrusion tasks’ trajectories in Rviz (Fig. 12).
Next, extra postprocessing and fine tuning can be performed in other programming platforms. In all the case studies in this work, a customized C# JSON file parser is implemented in Grasshopper (2018). The KUKAPRC package (Braumann and BrellCokcan 2011) and the Robots plugin (Soler 2018) are used to postprocess the trajectory into a KUKA Robot Language (KRL) file and ABB RAPID file for simulation and execution. The exported trajectory can be configured easily to work in other parametric design platforms and be adapted to other robotic simulation packages such as HAL (Schwartz 2012) and Jeneratiff (Dritsas 2015). As described in Sect. 2.2, such simulation tools are useful for visualizing a generated robotic motion plan and generates robotic brandspecific instruction code within the Grasshopper environment.
5 Case studies
Input model information, computation statistics, and fabrication time of the case studies
Model  Node count  Element count  Layer count  Sequence planning time (s)\(^{\dagger }\)  SC Cartesian planning time (s)  Retraction planning time (s)  Transition planning time (s)  Real extrusion time (h)  Size (mm) 

3D Voronoi (Sect. 5.1)  148  292  10  2759  1253  10  74 (RRT*)  3.2  150 
150  
320  
Topopt beam (small) (Sect. 5.2.1)  121  271  53  1650  1275  7  667 (STOMP)  3.6  400 
100  
100  
Topopt beam (large) (Sect. 5.2.2)  121  271  53  1847  1950  160  563 (STOMP)  –  2800 
700  
700 
5.1 3D Voronoi
Because of the Voronoigenerating algorithm, there is low variation in node valence and most nodes only connect four elements. Elements are well supported during each construction step and there are few very long elements. The internal topology does not have a trivial layerbased pattern. Thus, it is unintuitive for humans to find a sequence manually and the Choreo platform proves helpful.
However, elements at the boundary have smaller node valences and very long length. Even though the geometrically planned trajectory is feasible, the extruded element deviates from its ideal position because of the material’s thermal wrapping. This deviation is sometimes large enough that the robot is not able to connect to these elements in the subsequence extrusion processes by following the computed trajectories. This issue is resolved by adding micropath modifications to the computed Cartesian extrusion path in the postprocessing stage to extrude a “knot” at the node to compensate for the inaccuracy brought by the thermal behavior of the material.
5.2 Topologyoptimized simply supported beam
Using the ground structure topology optimization method described in Huang et al. (2018a), a simply supported beam was designed for the loads and boundary conditions shown in Fig. 15a, b. The resulting topology is fairly irregular when compared to a standardized mesh topology. The beam is scaled to a small size and a large size and two different machine setups are used in the planning. The largescale example is presented to demonstrate the potential of applying Choreo at the scale of a real building component, which in particular fits into the context of construction robotics.
5.2.1 Small scale
The smallscale beam spans 400 mm. A fixedbase KUKA KR6R900 (maximal reach 0.9 m) robot is used to execute the extrusion. The average element length of the model is fairly long and element length variation is low because the design is generated from a regular base mesh. However, the geometric configurations generated from these elements is not trivial. The trajectory highlighted in Fig. 15 shows the corresponding tool center point traveling trajectory from the transition planning result, indicating that the robot’s configuration changes significantly between many pairs of adjacent extrusions. As a result, trajectories that respect joint limits and avoid collisions are long and unintuitive to humans.
5.2.2 Large scale
6 Conclusion
6.1 Summary of contributions
This paper presents the first attempt to rigorously formulate the sequence and motion planning (SAMP) problem in the architectural robotic extrusion context using a CSP encoding. The research presented provides the first integrated algorithmic solution that uses a new hierarchical planning algorithm to tackle the extrusion SAMP problems with long planning horizons and complex geometry. Along with the planning algorithm, this paper also presents empirical computation results to provide insight into the communicational bottleneck that differentiates extrusion TAMP from the general discrete CSP problems. The presented algorithm and data serve as a comparing baseline for future algorithmic development.
This paper also presents a proofofconcept software implementation, called Choreo, which is a hardware and applicationagnostic. Choreo can be easily configured to work with industrial robots across brands, sizes, and joint spaces. Three case studies involving spatial trusses with nonstandard topologies, including two real fabrication experiments and one largescale assembly simulation, are presented to demonstrate the new fabrication possibilities enabled by applying the proposed planning framework.
6.1.1 Potential impact
The case studies presented in this work have demonstrated how the proposed extrusion planning framework’s integration into existing digital design workflow can support topology as a fundamental design variable on designers’ palette for robotic spatial extrusion. The emergence of this type of highlevel automated planning system can provide a better way for designers to interact with robots, shifting the machine programming experience back to highlevel tasks in the architectural language of shape and topology.
On the other hand, the flexibility and the efficiency of Choreo create a testbed for educators, researchers, and practitioners to explore the fabrication of 3D trusses across different topological and scale classes more boldly. It provides a general common ground for future research in extrusionoriented SAMP planner and creates a bridge between architectural robotics research community and task and motion planning research community.
6.1.2 Limitation and future work
One key limitation of the current work is on the need of human intervention for model decomposition to accelerate the sequence planner. An automatic decomposition algorithm will eliminate this last bit of human intervention and fully automate the planning process.
Potential directions of future investigation are summarized as follows:
Backtrack between planning layers When the planning system encounters a planning failure in any layer in the hierarchy, there is no backtracking mechanism provided to allow it to backtrack across planning layers. Thus, the hierarchical planning algorithm is not algorithmically complete for the entire system, i.e., guaranteed to find a solution if one exists, although the algorithm proposed in each level of the hierarchy is complete. Existing work in TAMP has devised various ways to allow this geometric backtracking across planning layers. The integration of some of this research is left as future work.
Extension to other robotic assembly applications All the algorithmic descriptions and case studies presented in this work are performed in the context of the specific application of robotic spatial extrusion. Spatial printing of 3D trusses can be viewed as assembling linear plastic beam elements in the space with melted plastic joint connection. Generalizing the proposed planning framework to a broader class of assembly applications, such as spatial positioning, requires little modification of the algorithm. These modifications include (1) different predefined plan skeletons, (2) different constraints on the end effector’s orientation in the semiconstrained Cartesian planner that takes the geometry of the end effector, element, and connection detail between elements into consideration. However, notice that this extension to spatial positioning applications does not include the automatic process for joining elements, but only includes the sequence and motion planning for a sequence of pick–transition–place–transition motions, assuming an external process handles the joining process flawlessly.
6.1.3 Concluding remark
Automatic sequence and motion planning (SAMP) has been a key missing link in the established digital designrobotic spatial extrusion workflow. Architectural robotic extrusion SAMP problems post unique technical challenges, including (1) long planning horizon and (2) structural stiffness, stability and geometric collision constraints for extrusion sequence planning. This paper presents a new algorithmic framework and a software tool called Choreo to fill in this missing link. The presented research clears away the technical barrier of the sequence and motion planning that has been congesting the workflow and limiting designbuild freedom. Although Choreo is still in its early stage, its flexibility and speed have already suggested an exciting future possibility: fabrication and assembly logic related to robotic constructibility could be integrated as a driver in iterative conceptual design, pushing the role of technical assessment from checking a nearly finalized design to an early stage design aid.
Footnotes
Notes
Acknowledgements
The authors want to acknowledge Thomas Cook, Khanh Nguyen, and Kodiak Brush at MIT for their work on constructing the earlystage prototype of the software and hardware presented in this work. The authors would also like to thank Jonathan Meyer from ROSIndustrial for his insightful discussion on GitHub, Zhongyuan Liu from University of Science and Technology of China for his help on generating the 3D Voronoi shape, Lavender Tessmer and Zhujing Zhang at MIT for their help on diagram drawing. The authors want to thank ArchiSolution Workshop (http://www.asworkshop.cn/) for their support on the designing and assembling of the mechanical extrusion system used in the case studies. Caelan Garrett acknowledges the support from NSF grants 1420316, 1523767 and 1723381, from AFOSR FA95501710165, from ONR grant N000141410486, and an NSF GRFP fellowship with primary award number 1122374. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the sponsors.
Compliance with ethical standards
Conflict of interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.
Supplementary material
References
 Alami R, Laumond JP, Siméon T (1994) Two manipulation planning algorithms. In: WAFR proceedings of the workshop on algorithmic foundations of robotics. A. K. Peters, Ltd., Natick, MA, USA, pp 109–125Google Scholar
 Baudisch P, Mueller S (2017) Personal fabrication. Found Trends Hum Comput Interact 10(3–4):165–293CrossRefGoogle Scholar
 BranchTech (2018) Branch Technology. https://www.branch.technology/. Accessed 14 Mar 2018
 Braumann J, BrellCokcan S (2011) Parametric robot control: integratedcad/cam for architectural design. In: Proceedings of the 31st annual conference of the association for computer aided design architecture (ACADIA)Google Scholar
 Cormen TH (2009) Introduction to algorithms. MIT Press, LondonzbMATHGoogle Scholar
 De Maeyer J, Moyaers B, Demeester E (2017) Cartesian path planningfor welding robots: Evaluation of the descartes algorithm. In: Proceedings of the 2017 22nd IEEE international conference on emerging technologies and factory automation (2017)Google Scholar
 Dechter R (2003) Constraint processing. Morgan Kaufmann, BurlingtonzbMATHGoogle Scholar
 Dogar M, Spielberg A, Baker S, Rus D (2015) Multirobot grasp planning for sequential assembly operations. In: Robotics and automation (ICRA), 2015 IEEE international conference on. IEEE, pp 193–200Google Scholar
 Dörfler K, Sandy T, Giftthaler M, Gramazio F, Kohler M, Buchli J (2016) Mobile robotic brickwork. In: Robotic fabrication in architecture, art and design 2016. Springer, pp 204–217Google Scholar
 Dornhege C, Eyerich P, Keller T, Trüg S, Brenner M, Nebel B (2012) Semantic attachments for domainindependent planning systems. Springer tracts in advanced robotics. Springer, pp 99–115Google Scholar
 Dritsas S (2015) A digital design and fabrication library. In: Proceedings of the symposium on simulation for architecture and urban design, pp 75–80Google Scholar
 Eversmann P, Gramazio F, Kohler M (2017) Robotic prefabrication of timber structures: towards automated largescale spatial assembly. Construct Robot 1(1–4):49–60CrossRefGoogle Scholar
 Fang FY (2017) Effect of print orientation on mechanical material behavior in fused deposition modeling 3D printing. Master’s Thesis, Massachusetts Institute of TechnologyGoogle Scholar
 FCL (2018) Flexible Collision Library. https://github.com/flexiblecollisionlibrary/fcl. Accessed 14 Nov 2018
 Gandia A, Parascho S, Rust R, Casas G, Gramazio F, Kohler M (2018) Towards automatic path planning for robotically assembled spatial structures. In: Robotic fabrication in architecture, art and design. Springer, pp 59–73Google Scholar
 Garrett CR, LozanoPérez T, Kaelbling LP (2018b) Samplingbased methods for factored task and motion planning. Int J Robot ResGoogle Scholar
 Garrett CR, LozanoPérez T, Kaelbling LP (2018a) Ffrob: leveraging symbolic planning for efficient task and motion planning. Int J Robot Res 37(1):104–136CrossRefGoogle Scholar
 Gelber MK, Hurst G, Bhargava R (2018a) Freeform assembly planning. arXiv:1801.00527
 Gelber MK, Hurst G, Comi TJ, Bhargava R (2018b) Modelguided design and characterization of a highprecision 3D printing process for carbohydrate glass. Addit Manuf 22:38–50CrossRefGoogle Scholar
 Giftthaler M, Sandy T, Dörfler K, Brooks I, Buckingham M, Rey G, Kohler M, Gramazio F, Buchli J (2017) Mobile robotic fabrication at 1:1 scale: the in situ fabricator. Construct Robot 1(1–4):3–14CrossRefGoogle Scholar
 Gilbert EG, Johnson DW, Keerthi SS (1988) A fast procedure for computing the distance between complex objects in threedimensional space. IEEE J Robot Autom 4(2):193–203. https://doi.org/10.1109/56.2083 CrossRefGoogle Scholar
 Gramazio F, Matthias K, Willmann J (2014) The robotic touch. Park Books, ZurichGoogle Scholar
 Grasshopper R (2018) Grasshopper rhinoceros. http://www.grasshopper3d.com/. Accessed 10 Jul 2018
 Hack N, Lauer WV (2014) Meshmould: robotically fabricated spatial meshes as reinforced concrete formwork. Arch Des 84(3):44–53Google Scholar
 Hauser K, Latombe JC (2010) Multimodal motion planning in nonexpansive spaces. Int J Robot Res 29(7):897–915CrossRefGoogle Scholar
 Hauser K, NgThowHing V (2011) Randomized multimodal motion planning for a humanoid robot manipulation task. Int J Robot Res 30(6):678–698CrossRefGoogle Scholar
 Helm V, Willmann J, Thoma A, Piškorec L, Hack N, Gramazio F, Kohler M (2015) Iridescence print: robotically printed lightweight mesh structures. 3D Print Addit Manuf 2(3):117–122CrossRefGoogle Scholar
 Helmert M (2006) The fast downward planning system. J Artif Intell Res 26:191–246CrossRefGoogle Scholar
 Huang Y, Zhang J, Hu X, Song G, Liu Z, Yu L, Liu L (2016) Framefab: robotic fabrication of frame shapes. ACM Trans Graph 35(6):224Google Scholar
 Huang Y, Carstensen J, Mueller C (2018a) 3D truss topology optimization for automated robotic spatial extrusion. In: Proceedings of the international association for shell and spatial structures (IASS) symposium 2018Google Scholar
 Huang Y, Carstensen J, Tessmer L, Mueller C (2018b) Robotic extrusion of architectural structures with nonstandard topology. In: Robotic fabrication in architecture, art and design 2018. SpringerGoogle Scholar
 IKFast (2018) Ikfast: the robot kinematics compiler. http://openrave.org/docs/0.8.2/openravepy/ikfast/. Accessed 22 Jul 2018
 Kalakrishnan M, Chitta S, Theodorou E, Pastor P, Schaal S (2011) Stomp: stochastic trajectory optimization for motion planning. In: 2011 IEEE international conference on robotics and automation, pp. 4569–4574Google Scholar
 Karaman S, Frazzoli E (2011) Samplingbased algorithms for optimal motion planning. arXiv:1105.1186
 Kavraki LE, Svestka P, Latombe JC, Overmars MH (1996) Probabilistic roadmaps for path planning in highdimensional configuration spaces. IEEE Trans Robot Autom 12(4):566–580CrossRefGoogle Scholar
 Krontiris A, Bekris K (2015) Dealing with difficult instances of object rearrangement. In: Robotics: science and systems (RSS). https://doi.org/10.15607/RSS.2015.XI.045
 Krontiris A, Bekris KE (2016) Efficiently solving general rearrangement tasks: a fast extension primitive for an incremental samplingbased planner. In: International conference on robotics and automation (ICRA), IEEE, pp 3924–3931Google Scholar
 Lagriffoul F (2016) On benchmarks for combined task and motion planning. In Robotics: science and systems (RSS) 2016 workshop on task and motion planningGoogle Scholar
 Lagriffoul F, Andres B (2016) Combining task and motion planning: a culprit detection problem. Int J Robot Res 35(8):890–927CrossRefGoogle Scholar
 Lagriffoul F, Dimitrov D, Bidot J, Saffiotti A, Karlsson L (2014) Efficiently combining task and motion planning using geometric constraints. Int J Robot Res 33(14):1726–1747CrossRefGoogle Scholar
 LaValle SM (2006) Planning algorithms. Cambridge University Press, CambridgeCrossRefGoogle Scholar
 LozanoPérez T, Kaelbling LP (2014) A constraintbased method for solving sequential manipulation planning problems. In: Intelligent robots and systems (IROS 2014), 2014 IEEE/RSJ international conference on. IEEE, pp 3684–3691Google Scholar
 Makhal A, Goins AK (2017) Reuleaux: robot base placement by reachability analysis. arXiv:1710.01328
 Mataerial (2018) Mataerial. http://www.mataerial.com/. Accessed 31 Oct 2018
 McDermott D, Ghallab M, Howe A, Knoblock C, Ram A, Veloso M, Weld D, Wilkins D (1998) Pddl—the planning domain definition language. Tech. rep., Yale Center for Computational Visionand ControlGoogle Scholar
 McGuire W, Gallagher R, Ziemian R (1999) Matrix structural analysis. Wiley, OxfordGoogle Scholar
 Mueller S, Im S, Gurevich S, Teibrich A, Pfisterer L, Guimbretière F, Baudisch P (2014) Wireprint: 3D printed previews for fast prototyping. In: Proceedings of the 27th annual ACM symposium on user interface software and technology. ACM, pp 273–280Google Scholar
 Oxman N, Laucks J, Kayser M, Tsai E, Firstenberg M (2013) Freeform 3D printing: towards a sustainable approach to additive manufacturing. Green design, materials and manufacturing processes, p 479Google Scholar
 Parascho S, Gandia A, Mirjan A, Gramazio F, Kohler M (2017) Cooperative fabrication of spatial metal structures. In: Fabricate 2017. UCL Press, pp 24–29Google Scholar
 Peng H, Briggs J, Wang CY, Guo K, Kider J, Mueller S, Baudisch P, Guimbretière F (2018) Roma: interactive fabrication with augmented reality and a robotic 3D printer. In: Proceedings of the 2018 CHI conference on human factors in computing systems. ACM, pp 579:1–579:12Google Scholar
 Peng H, Wu R, Marschner S, Guimbretière F (2016) Onthefly print: incremental printing while modelling. In: Proceedings of the 2016CHI conference on human factors in computing systems. ACM, pp 887–896Google Scholar
 Phear JB (1850) Elementary mechanics. MacMillan, CambridgeGoogle Scholar
 Piker D (2018) Kangaroo physics. https://www.food4rhino.com/app/kangaroophysics. Accessed 26 Jul 2018
 Quigley M, Conley K, Gerkey B, Faust J, Foote T, Leibs J, Wheeler R, Ng AY (2009) Ros: an opensource robot operating system. In: ICRA workshop on open source software, vol. 3. Kobe, Japan, p 5Google Scholar
 Ratliff N, Zucker M, Bagnell JA, Srinivasa S (2009) Chomp: gradient optimization techniques for efficient motion planning. In: International conference on robotics and automation. IEEE, pp 489–494Google Scholar
 ROSI (2018) ROS industrial—descartes. https://github.com/rosindustrialconsortium/descartes (2018a). Accessed 14 Mar 2018
 ROSI (2018) ROS industrial—Godel. https://github.com/rosindustrialconsortium/godel (2018b). Accessed 14 Mar 2018
 ROSI (2018) ROS IndustrialIndustrial Moveit. https://github.com/rosindustrial/industrial_moveit (2018c). Accessed 14 Mar 2018
 Russell SJ, Norvig P (2016) Artificial intelligence: a modern approach. Pearson Education Limited, MalaysiazbMATHGoogle Scholar
 Schulman J, Duan Y, Ho J, Lee A, Awwal I, Bradlow H, Pan J, Patil S, Goldberg K, Abbeel P (2014) Motion planning with sequential convex optimization and convex collision checking. Int J Robot Res 33(9):1251–1270CrossRefGoogle Scholar
 Schwartz T (2012) Hal: extension of a visual programming language to support teaching and research on robotics applied to construction. In: Robotic fabrication in architecture, art and design 2012. Springer, pp 92–101Google Scholar
 Siméon T, Laumond JP, Corts J, Sahbani A (2004) Manipulation planning with probabilistic roadmaps. Int J Robot Res 23(7–8):729–746CrossRefGoogle Scholar
 Soler V (2018) Robots plugin for grasshopper. https://github.com/visose/Robots. Accessed 20 Jun 2018
 Soler V, Retsin G, Jimenez Garcia M (2017) A generalized approach to nonlayered fused filament fabrication, pp 562–571Google Scholar
 Søndergaard A, Amir O, Eversmann P, Piškorec L, Stan F, Gramazio F, Kohler M (2016) Topology optimization and robotic fabrication of advanced timber spaceframe structures. In: Robotic fabrication in architecture, art and design 2016. Springer, pp 190–203Google Scholar
 Stilman M, Kuffner JJ (2008) Planning among movable obstacles with artificial constraints. Int J Robot Res 27(11–12):1295–1307CrossRefGoogle Scholar
 Stilman M, Schamburek JU, Kuffner J, Asfour T (2007) Manipulation planning among movable obstacles. In: Proceedings 2007 IEEE international conference on robotics and automation, pp 3327–3332Google Scholar
 Sucan IA, Chitta S (2018) Moveit! http://moveit.ros.org. Accessed 14 Mar 2018
 Tam KM, Marshall DJ, Gu M, Kim J, Huang Y, Lavallee JA, Mueller CT (2018) Fabricationaware structural optimisation of lattice additivemanufactured with robotarm. Int J Rapid Manuf 7:2–3Google Scholar
 Toussaint M (2015) Logicgeometric programming: an optimization based approach to combined task and motion planning. In: Proceedings of the 24th international conference on artificial intelligence, IJCAI’15. AAAI Press, pp 1930–1936Google Scholar
 Wu R, Peng H, Guimbretière F, Marschner S (2016) Printing arbitrary meshes with a 5dof wireframe printer. ACM Trans Graph 35(4):101Google Scholar
 Yuan PF, Meng H, Yu L, Zhang L (2016) Robotic multidimensional printing based on structural performance. In: Robotic fabrication in architecture, art and design 2016. Springer, pp 92–105Google Scholar
 Yu L, Huang Y, Liu Z, Xiao S, Liu L, Song G, Wang Y (2016) Highly informed robotic 3D printed polygon mesh: a novel strategy of 3D spatial printing. In: Proceedings of the 36st annual conference of the association for computer aided design in architecture (ACADIA), pp 298–307Google Scholar