Force Perception (in Robotics)
Prerequisites
- Read Kinematics and Dynamics
- Read Sensors and Sensing page
- Basics of electrical circuits (resistance, capacitance, voltage-divider, etc.)
- Basics of mechanics (Hooke’s Law, stress-strain curve, etc.)
General Motivation
Robots are expected to interact closely and safely with humans as well as with their environment. To achieve this, there is one sense that humans use all the time but is often neglected in robotics: the sense of touch.
In robotics, touch refers to situations where a robot gets into physical interaction with another real-world agent, such as a human, an object, or even another robot. During that interaction, forces are exchanged between two agents. This chapter focuses on how to measure these interaction forces.
Same as humans, robots can perceive interaction forces at different levels. When lifting an object, humans first sense the overall force, such as its weight, then acquire more fine-grained information through touch, such as local pressure distribution. This distinction leads to the use of force sensing for global interaction forces and tactile sensing for localized contact information in robotics. But more about this later.
Let us first have a look at some examples. Below are some illustrations and explanations of the three main interaction categories in which touch is used in robotics: manipulation, exploration and reaction.
Manipulation
$\Rightarrow$ Interaction between a robot (active agent) and an object (passive agent).
During manipulation, the robot senses the object and adapts its actions to control it accordingly. Object manipulation is essential in fields like industrial robotics. In tasks like grasping, force perception is used to estimate the grasp force applied by the robotic hand, to prevent damaging the object or slipping. More about grasping can be found on its dedicated page (click here).
Key points of the video:
A robotic hand manipulates a water-filled champagne glass. As the hand tilts the glass, the moving water changes the internal mass distribution, creating disturbances during manipulation. Using tactile sensors at the fingertips, the robotic fingers adapt their movement and contact forces to control the glass despite these disturbances. In the end of the sequence, a larger motion causes the water to spill.
Exploration
$\Rightarrow$ Interaction between a robot (active agent) and an object (passive agent).
During exploration, the robot performs movements to discover unknown properties of an object, without the objective of directly controlling it. Touch can be used to determine material properties such as softness, surface texture, shape, temperature or sometimes even the friction coefficient. For example, it is possible to determine whether an object is stiff or compliant, smooth or rough.
Key points of the video:
A humanoid robot explores different objects by moving its fingers along their surface, trying to identify them through touch. First it explores a bottle, then a jar, a telephone handset and finally a glass. Tactile sensors at the robot’s fingertips provide contact information as the fingers slide over the object. These contact points are accumulated into a noisy point cloud, which is then used to reconstruct an approximate shape of the object. This type of tactile exploration is useful in situations where vision is unavailable, for example due to bad lighting conditions.
Reaction
$\Rightarrow$ Interaction between a robot (active agent) and another active agent (human/robot).
During reaction interactions, the robot continuously acts, perceives and adapts in real time based on the feedback it receives from the other agent. This enables, for example, safe operation of robots around humans: the robot can detect abnormal contact and adjust its movement to avoid harm. In the field of haptics and more specifically teleoperation, touch also allows humans to guide robots while receiving force feedback. More on haptics can be found on the dedicated page (click here).
Key points of the video:
A robotic hand is shown holding continuous contact with a fake human arm. The arm is moved by a person, making it a dynamic interaction involving another active agent. A force sensor located at the robot’s end-effector measures the interaction forces, which allows the robotic hand to adapt its movement continuously. In the end of the video, we can see that the robot reacts differently to fast disturbing contacts: non-consistent disturbances are rejected. This shows that the robot can distinguish intentional human guidance from accidental disturbances.
As you can imagine, force perception is used in many areas. From biomedical robotics such as surgical robots, to rehabilitation systems like exoskeletons and humanoid robots.
Before moving on, try the quiz below.
Quiz
Course Content
Now that we have seen why robots need to perceive forces, we can dive into how force perception is implemented. As introduced earlier, robots can perceive interaction forces at different levels of detail, from global interaction forces (force sensing or intrinsic sensing) to more fine-grained contact information (tactile sensing or extrinsic sensing).
The distinction between intrinsic and extrinsic sensing is based on the location of the sensors on the robot (see figure below). Intrinsic sensing, relies on sensors placed within the mechanical structure of the robot. On the other hand, extrinsic sensing, refers to sensors mounted directly at the robot’s contact surface.
In summary:
-
Force sensing (intrinsic) measures the global forces and torques applied to the system at a specific point, considered infinitesimally small. It captures the overall push, pull and twist experienced by the robot at that location, usually at a joint or a structural element.
-
Tactile sensing (extrinsic) measures stress or pressure distributions over a surface rather than at a single point. It relies on an array of sensing elements, forming what can be thought of as an electronic skin. Because multiple contact points are available, tactile sensing can detect slippage and precise contact location.
On this page, we will move gradually from force sensing, which describes interactions occurring at a single point, to tactile sensing, where sensing extends across a bigger surface. Although the examples shown in the introduction mainly focused on hands and fingertips, tactile sensing can be applied to the entire body of a robot. However, challenges such as wiring complexity and limited mechanical flexibility must also be addressed. We will see that later.
-
Section 2.2.3.1: Force Sensing
Introduction to force and torque sensing methods.
Force Sensing
Let us begin with a quick reminder of forces and torques (also called moments) we want to measure. Force is given in Newtons [N] and produces linear movement, whereas torque is given in Newton-meters [Nm] and produces rotational movement. They are both vector quantities defined in 3D space, meaning they can be decomposed into components of the orthonormal basis of $\mathbb{R}^3$ (x, y, and z axis).
$$\text{Force: } \mathbf{F} = (F_{x}, F_{y}, F_{z}) \in \mathbb{R}^3$$ $$\text{Torque: } \mathbf{M} = (M_{x}, M_{y}, M_{z}) \in \mathbb{R}^3$$
The goal of force-torque (F/T) sensing is to obtain a complete description of all forces and torques exchanged at the contact location. This is possible because the sum of all forces and the sum of all moments acting on the system must be equal to zero (in static equilibrium):
$$ \sum \mathbf{F} = 0 \qquad \sum \mathbf{M} = 0 $$
At this point, we can introduce the wrench vector $W$. It represents the complete mechanical interaction at the contact point, combining the force and torque vectors into a single vector in $\mathbb{R}^6$. The wrench is the unknown quantity to be determined in F/T sensing.
$$ W = \begin{bmatrix} F_c \cr M_c \end{bmatrix} \in \mathbb{R}^6, \qquad F_c = \begin{bmatrix} F_x \cr F_y \cr F_z \end{bmatrix}, \qquad M_c = \begin{bmatrix} M_x \cr M_y \cr M_z \end{bmatrix} $$
Where $F_c$ and $M_c$ are the force and moment vectors applied at the contact location.
As described earlier, F/T sensing in robotics relies on intrinsic sensors, which are embedded within the robot’s structure. In this chapter, we will have a look at two different F/T sensing approaches: first, dedicated F/T sensors that directly measure these quantities, and second, sensorless methods that estimate forces and torques indirectly.
Try the quiz to make sure that you understood forces and torques.
Quiz
A) Force/Torque Sensors
Force-torque sensors are classified based on the number of axes (or degrees of freedom DOF) they measure.
-
Three-Dimensional Force Sensors (3DOF):
These sensors measure only forces, not torques. They provide information about the three translational force components along the $x$, $y$ and $z$ axes. The corresponding wrench vector is:$$ W = [F_x, F_y, F_z]^T \in \mathbb{R}^3 $$
3DOF sensors are used when only translational forces matter. They are often mounted near the end-effector and can for example measure the weight of an object or detect simple contact with a surface.
-
Six-Dimensional Force/Torque Sensors (6DOF):
These sensors measure both forces and torques, covering the three translational and three rotational axes. Besides pushing or pulling forces, they can also measure bending and twisting effects. The wrench vector they measure is:$$ W = [F_x, F_y, F_z, M_x, M_y, M_z]^T \in \mathbb{R}^6 $$
Sensing Principle and Mechanical Implementation:
The sensing principle of F/T sensors relies on detecting strain (deformation) in an elastic structure. When a force is applied, the elastic structure deforms and this deformation is converted into an electrical signal using strain gauges. By measuring the strain in the structure, the applied force can be determined using Hooke’s law. The detail of the mathematical model is shown later.
Below are examples of elastic structures used in F/T sensors:
1) Cross-Beam Structure:
The elastic base is shaped like a crossbar, consisting of an inner ring (central hub) connected to the fixed outer ring by flexible supporting beams. The whole piece is machined out of a single piece of material, to ensure high stiffness and to avoid hysteresis.
When load is applied to the central hub, the beams deform depending on the force direction. Vertical forces ($F_z$) cause the beams to bend up or down, while shear forces and torques induce a complex combination of tension and compression across the different arms of the cross. Below we can observe the effect of a horizontal force $Fx$ and a torque $Mz$ acting on the central hub.
Strain gauges are directly bonded on the surfaces of the beams to measure their strain.
2) Parallel Structure (Stewart Platform):
This structure consists of an upper mobile platform and a lower fixed base connected by six legs. An example of a miniature stewart platform is shown in the next figure.
This design distributes the applied load through the structure, causing primarily axial strain (tension or compression) along the longitudinal axis of the limbs. Strain gauges are bonded on both sides (front and back) of the legs connecting the two platforms to measure the strain.
Mathematical Model:
From the electrical signal obtained from the strain gauges, we can determine the forces and torques acting on the sensor.
Let us first recall Hooke’s law in its one-dimensional form. It is stating that for an elastic element, the applied force is proportional to its deformation (like a spring):
$$ F = k \cdot u $$
where $F$ is the applied force, $u$ is the deformation and $k$ is the stiffness of the elastic element.
Assumption: The elastic structure stays within the linear region of the stress–strain curve, where Hooke’s law applies. This linear relationship is assumed in the mathematical model.
By extending Hooke’s law to multiple dimensions, we can write the relationship between the applied wrench $W$ and the measured strain vector $u$ as follows:
$$ W = K \cdot u $$
where:
- $W \in \mathbb{R}^6$ is the wrench vector (combining forces and torques)
- $K \in \mathbb{R}^{6 \times n}$ is the stiffness matrix of the elastic structure
- $u = [u_1, u_2, \dots, u_n]^T$ is the vector of strain measurements
- $n \geq 6$ is the number of strain sensing points (strain gauges)
The stiffness matrix $K$ depends on the geometry and material properties of the elastic structure.
In practice, $K$ is often determined experimentally during a calibration procedure, by applying known loads to the sensor and measuring the resulting strain.
B) Sensorless Force/Torque Estimation
It is also possible to determine external forces and torques without embedding dedicated sensors. This sensorless method relies on the robot’s internal data (available without special hardware), such as the amount of current drawn by its motors. In most motors, the generated torque is proportional to the motor current. By comparing the actual torque output (derived from current) with the theoretically required torque, it is possible to determine the existence and magnitude of an external force.
Below, we look at two different approaches to estimate external forces using motor current: model-based and model-free (Neural Network–based).
Approach A Model-Based Estimation
This approach is called model-based, as it uses the robot’s dynamics and kinematics (= model) to compute the external force applied on the robot. If you need a quick reminder about the Jacobian, have a look at the Kinematics course.
Contact force estimation follows three steps:
- Estimate the external joint torques induced by the force
- Compute the Jacobian at the contact location
- Convert the joint torques into cartesian force
1) Estimating the external joint torque $\tau_{\text{ext}}$
We start with the basic Lagrangian expression of the robot’s dynamics (seen previously in the Dynamics course):
$$ M(\theta)\ddot{\theta} + C(\theta,\dot{\theta})\dot{\theta} + g(\theta) = \tau $$
Where:
- $\theta \in \mathbb{R}^n$: vector of joint variables (position)
- $\ddot{\theta}, \dot{\theta}$: joint acceleration and velocity
- $M(\theta)$: mass matrix
- $C(\theta,\dot{\theta})\dot{\theta}$: vector accounting for Coriolis or centrifugal torques
- $g(\theta)$: vector of gravity torques
- $\tau$: vector of torques applied by the robot’s motors
To properly estimate external forces, we use and extended dynamic model, which includes the external torques:
$$ M(\theta)\ddot{\theta} + C(\theta,\dot{\theta})\dot{\theta} + g(\theta) = \tau + \tau_{\text{ext}} $$
Where:
- $\tau_{\text{ext}}$: vector of induced torque by the external contact force
By computing the difference between the theoretical model torque and the measured motor torque, we finally obtain the external joint torque $\tau_{\text{ext}}$.
2) Computing the Jacobian at the contact location $J_c$
Once $\tau_{\text{ext}}$ is estimated, the next step is to determine how a force at the contact point affects the joints. As shown in the figure below, a contact may occur on a link (panel (a)) or on the end-effector (panel (b)). The contact point determines which joints are affected.
The position of the contact point relative to joint $i$, on which the contact is happening, is obtained by subtracting the absolute position of the origin of link $i$, denoted $p_i(\theta)$, from the absolute position of the contact point on that link, denoted $p_{c}(\theta)$:
$$ p_{i,c}(\theta) = p_{c}(\theta) - p_i(\theta) $$
Having this, the contact-point Jacobian $J_c(\theta)$ can be computed. It is derived from the link Jacobian $J_i(\theta)$:
$$ J_c(\theta) = \begin{bmatrix} I & -\,S(p_{i,c}(\theta)) \cr 0 & I \end{bmatrix} J_i(\theta) $$
where $S(\cdot)$ is the skew-symmetric matrix, composed of the components of the vector $p_{i,c}(\theta)$:.
This Jacobian describes how a force applied at the contact point generates joint torques.
3) Computing the contact force
Finally we can compute the wrench $W \in \mathbb{R}^6$ by resolving the following equation:
$$ \tau_{\text{ext}} = J_c^T(\theta) W $$
Approach B Model-Free Estimation (Neural Network Based)
The second proposed approach is machine learning based and does not rely on any physics equation. Instead of using a model, the wrench vector $W \in \mathbb{R}^6$ is determined by a neural network (NN). To train the NN, this approach needs real-world data, that can be collected using an actual F/T sensor. Data is usually obtained through learning from demonstration, a method whereby an operator passively moves the robot to show how to perform a given task. Data on F/T perception are gathered as the robot makes various contacts with the environment, see course on learning from demonstration.
The variables fed to the NN are the robot’s internal state signals, such as joint currents, joint positions $\theta$, joint velocities $\dot{\theta}$ and joint accelerations $\ddot{\theta}$. All these inputs are put together into one input vector $x_n$.
For example, for an $n$-joint robot, an input vector may look like:
$$ x_n = \begin{bmatrix} I_1, \ldots, I_n, \
\theta_1, \ldots, \theta_n, \
\dot{\theta}_1, \ldots, \dot{\theta}_n, \
\ddot{\theta}_1, \ldots, \ddot{\theta}_n \end{bmatrix}^T $$
At every instant, the NN takes the input signal vector $x_n$ and outputs the estimated external wrench vector $W$ in real time:
$$ W = NN(x_n) $$
Where $NN(\cdot)$ represents the trained neural network structure.
An example of such a neural network is shown in the figure below.
The performance of this approach can be seen in the video below.
Key points of the video:
In the first part, the end-effector of the robot gets in contact with a surface and therefore external forces are created. On the plot, we can see that the estimated forces closely match the measurements obtained from the built-in F/T sensor. In the second part, an application example is shown in which the pin is inserted into a corresponding hole. During this task, the F/T sensor is disabled and F/T feedback is only given by the estimator. In the last part, the authors show that sensorless force estimation can also be used for human guidance.
To conclude sensorless F/T estimation, here are its main advantages and disadvantages:
| Advantages | Disadvantages |
|---|---|
| No dedicated F/T sensor needed | Dependence on model accuracy (model-based approach) |
| Reduced hardware complexity | Dependence on training data (model-free approach) |
| Lower system cost | Performance degrades in unmodeled or untrained situations |
For additional information and another example of sensorless F/T estimation, feel free to read the paper linked below.
Further Reading: Intrinsic sense of touch
In this paper, the authors present a sensorless intrinsic sense of touch method, that goes one step further as the force estimation we saw before. The robot is able not only to estimate interaction forces, but also to localize the contacts and reconstruct touch trajectories over its body. This enables interactions like virtual buttons, writing on the robot surface and more intuitive physical human–robot interaction. It is showing that sensorless estimation can also provide tactile feedback and not only force feedback. A video is also available by following the link.
Intrinsic sense of touch for intuitive physical human–robot interaction
M. Iskandar, A. Albu-Schäffer and A. Dietrich
Before moving on, answer the questions about force sensing in the quiz below.
Quiz
Which measurement output is corresponding to that scenario? (single answer possible)
Which of the following statements is correct? (single answer possible)
Which solution is the most appropriate to still estimate interaction forces? (single answer possible)
What is the most likely reason for this behavior? (single answer possible)
Credits
This page was created by Mael Studer, under the supervision of Prof. Aude Billard.
This page used the following resources:
-
Handbook of Robotics, Springer (Chapter 19. Force and Tactile Sensors)
Additional Resources
-
Intrinsic sense of touch for intuitive physical human-robot interaction (M. Iskandar, A. Albu-Schäffer and A. Dietrich)
-
Measurement Methods for Capacitances in the Range of 1 pF–1 nF: A Review (O. Kanoun et al.)