Soft active materials can generate flexible locomotion and change configurations through large deformations when subjected to an external environmental stimulus. They can be engineered to design 'soft machines' such as soft robots, compliant actuators, flexible electronics, or bionic medical devices. By embedding ferromagnetic particles into soft elastomer matrix, the ferromagnetic soft matter can generate flexible movement and shift morphology in response to the external magnetic field. By taking advantage of this physical property, soft active structures undergoing desired motions can be generated by tailoring the layouts of the ferromagnetic soft elastomers. Structural topology optimization has emerged as an attractive tool to achieve innovative structures by optimizing the material layout within a design domain, and it can be utilized to architect ferromagnetic soft active structures. In this paper, the level-set-based topology optimization method is employed to design ferromagnetic soft robots (FerroSoRo). The objective function comprises a sub-objective function for the kinematics requirement and a sub-objective function for minimum compliance. Shape sensitivity analysis is derived using the material time derivative and adjoint variable method. Three examples, including a gripper, an actuator, and a flytrap structure, are studied to demonstrate the effectiveness of the proposed framework.