In this project a method is proposed for automatic loading of objects on a swinging conveyor trolley with an industrial robot. The robot tracks the swinging motion of the conveyor trolley using computer vision.
An industrial robot manipulator picks up objects and places them on the hanging trolley. It is assumed that the conveyor belt can be stopped temporarily during loading. The main challenge is therefore to compensate for the swinging motion of the trolley and for the unknown orientation at the loading station.
Laboratory set-up
An experimental set-up was built at the laboratory at the Department of Production and Quality Engineering at NTNU. It consists of a conveyor line in the roof, two cameras and a robot. The conveyor is of the type power & free, which means that individual trolleys can stop at stations without stopping the entire conveyor line.
The two cameras continuously delivers images to a computer over Ethernet. The images are processed on a computer to generate set-points for control of the robot. The robot is a UR5 from Universal Robots.
Kinematics
The conveyor trolleys used in the experiments are welded from steel cylinders or steel bars. This makes it natural use a visual model based on edge detection for visual tracking. The W frame is stationary while the B frame is attached to the swinging trolley. It is necessary to estimate the rotation matrix from the W frame to the B frame in order to control the robot.
A particle filter is used to estimate the rotation matrix using the continuous stream of images from the two cameras. The cameras deliver 1024 x 768 pixels 8-bit images at 35 Hz
Loading panels on a swinging conveyor trolley
In this experiment the goal is to demonstrate delivery of panels on the swinging trolley. A vacuum gripper is mounted on the robot end-effector in order to grab the panel.
The videos below demonstrates the proposed method.
Loading cylindrical objects on the conveyor trolley
In this experiment the robot delivers cylindrical objects on the swinging trolley.
In this experiment the robot executes the following motion:
- The robot first moves to the initial position shown.
- The robot then starts to move towards the final position, which is specified relative to the moving body frame.
- When the robot reaches the final position it moves back to the initial position, where it stops.
The video below demonstrates the proposed method.
Detection of panels on the conveyor trolley
Parts of the trolley becomes occluded when panels are loaded on the trolley. In this experiment this information is used to determine whether panels are loaded onto the trolley. The method is described in the paper:
Myhre, Torstein A., Aksel A. Transeth, and Olav Egeland. "Loading of hanging trolleys on overhead conveyor with industrial robots." Technologies for Practical Robot Applications (TePRA), 2015 IEEE International Conference on. IEEE, 2015.
The video below shows a sequence where a panel is moved between the two loading positions on the trolley.
Estimation of the mass center position
As mentioned above, the W frame is stationary while the B frame is attached to the swinging trolley. The kinematics of the trolley is described using Euler angles, as shown in the image.
The ϕ parameters are dynamic, and must be estimated continuously. The θ parameters are static and only have to be estimated once. The particle filter is designed for estimating dynamic parameters only, but accurate knowledge of the static parameters is required to achieve good performance in this application. A method that extends the particle filter with the ability to estimate static parameters was therefore used. The method is presented in the paper:
Myhre, Torstein A., and Olav Egeland. "Parameter estimation for visual tracking of a spherical pendulum with particle filter." Multisensor Fusion and Integration for Intelligent Systems (MFI), 2015 IEEE International Conference on. IEEE, 2015.
In the video below the proposed method estimates the vector of static parameters θ, i.e. the parameters that describes the position of the mass center of the trolley.
