Design generation requires tight integration of neural and symbolic reasoning, as good design must meet explicit user needs and honor implicit rules for aesthetics, utility, and convenience. Current automated design tools driven by neural networks produce appealing designs, but cannot satisfy user specifications and utility requirements. Symbolic reasoning tools, such as constraint programming, cannot perceive low-level visual information in images or capture subtle aspects such as aesthetics. We introduce Spatial Reasoning Integrated Generator (SPRING) for design generation. SPRING embeds a neural and symbolic integrated spatial reasoning module inside the deep generative network. The spatial reasoning module samples the set of locations of objects to be generated from a backtrack-free distribution. This distribution modifies the implicit preference distribution, which is learned by a recursive neural network to capture utility and aesthetics. Sampling from the backtrack-free distribution is accomplished by a symbolic reasoning approach, SampleSearch, which zeros out the probability of sampling spatial locations violating explicit user specifications. Embedding symbolic reasoning into neural generation guarantees that the output of SPRING satisfies user requirements. Furthermore, SPRING offers interpretability, allowing users to visualize and diagnose the generation process through the bounding boxes. SPRING also handles novel user specifications not encountered during its training with zero-shot constraint transfer. Quantitative evaluations and a human study show that SPRING outperforms baseline generative models, delivering high design quality and better meeting user specifications.

Continual learning requires the ability to reliably transfer previously learned knowledge to new tasks without disrupting established competencies. Methods such as Progressive Neural Network [1] accomplish high-quality transfer learning while nullifying the insidious problem of catastrophic forgetting. However, most module-based continual learning systems require task labels during operation – a constraint that limits their application in many real-world conditions where task indicators are opaque. This paper proposes a task detector neural algorithm to acquire task information while maintaining immunity to forgetting. Our proposed task detector allows progressive neural networks (and many similar systems) to operate without task labels during test time. Our task detector is built from familiarity autoencoders which recognize the nature of the required task from input data. We demonstrate the generality and effectiveness of our approach through experiments in video game playing and automated image repair. Our results show near-perfect task recognition in all domains (>.99 F1), rewards above published single-task scores in MinAtar, and realistic image repairs on damaged human face pictures. The performance of our integrated method is nearly identical to the progressive systems equipped with ground-truth task labels1