On-site ergonomic prevention aims at decreasing work-related musculoskeletal disorders in the industry. Recognition of hazardous posture during the execution of manual tasks by humans moves towards this direction by helping to classify those tasks as potential assignments to a robot. Sensor-driven motion capture using body-mounted inertial sensors associated with pattern recognition enables to track joints all along with a hazardous task. In this work, different deep neural networks were compared for time series classification of motion primitives based on the Ergonomic Assessment Worksheet. Moreover, to investigate the possibility of using a reduced number of sensors for the recognition, the performance achieved using only an optimal set of sensors is contrasted as well. This optimal set of sensors was formed using a stochastic-biomechanic approach, the Gesture Operational Model. Compared to all-sensor configuration, the best performance was achieved using the optimal set of sensors and the Rocket transformation.