The movie "The Lord of the Rings", the plush bear in "Teddy Bear", the tribal princess in "Avatar"... The vivid performances of the classic avatars in the movie always impress the audience, and they are given The back of life comes from an important technology technology - motion capture.
Motion capture, referred to as Mocap, refers to the technique of recording and processing the movements of people or other objects. After multiple cameras capture the actions of real actors, these actions are restored and rendered to the corresponding avatar. The technical use of this process is motion capture, which is expressed in English as Motion Capture.
Motion capture technology involves direct measurement of data processed by the computer in terms of dimensional measurements, positioning of objects in the physical space, and orientation measurements. A tracker is set at a key part of the moving object, and the position of the tracker is captured by the Motion capture system, and then processed by a computer to obtain data of the three-dimensional space coordinates. When the data is recognized by the computer, it can be applied in the fields of animation, gait analysis, biomechanics, ergonomics and so on.
Background of motion capture technology
The origin of motion capture is generally considered to be the rotoscope that Fleischer invented in 1915. This is a technique that has been produced in animation production. The artist simulates the realistic performance of an animated character in a virtual world by finely depicting each frame of static pictures in their live-action video clips.
The process itself is boring. But for these animators, fortunately and memorably, in 1983, the Massachusetts Institute of Technology (MIT) developed a set of graphical marionettes.
The system uses an early optical motion capture system called Op-Eye, which relies on a series of LEDs to generate animation scripts by creating actions (Sturman, 1999). In essence, this marionette served as the first "motion capture costume." It comes with a very limited number of sensor balls that can roughly locate the key skeletal points of the anatomy.
The emergence of this technology quickly laid the foundation for the rapid development of motion capture, providing a trail for subsequent motion capture, and also led the trend of motion capture technology, including today's motion capture technology.
Basic principles of motion capture technology
Motion capture system refers to professional technical equipment used to implement motion capture. Different motion capture systems follow different principles and the system components are different.
In general, motion capture systems usually consist of two major parts: hardware and software. The hardware generally includes signal transmitting and receiving sensors, signal transmission devices, and data processing devices; the software generally includes functional modules such as system settings, spatial positioning calibration, motion capture, and data processing.
The signal transmitting sensor is usually located at a key part of the moving object, such as the joint of the human body. After the continuous signal is received by the positioning sensor, the signal is sent to the data processing workstation through the transmission device, and the motion solution is calculated in the software to obtain the coherent three-dimensional motion data, including The three-dimensional space coordinates of the moving target, the six-degree-of-freedom motion parameters of the human joint, and the three-dimensional skeleton motion data can be used to drive the skeletal animation, which is the general workflow of the motion capture system.
Motion capture technology
sensor
The so-called sensor is a tracking device fixed to a specific part of the moving object, which will provide the motion capture system with the position information of the moving object motion, and generally determines the number of trackers according to the degree of detail of the capturing.
Signal capture device
This type of device will vary depending on the type of motion capture system, which is responsible for the capture of position signals. For mechanical systems, it is a circuit board that captures electrical signals, and for optical motion capture systems, it is a high-resolution infrared camera.
Data transmission equipment
The motion capture system, especially the motion capture system that requires real-time effects, requires a large amount of motion data to be quickly and accurately transmitted from the signal capture device to the computer system for processing, and the data transmission device is used to do this.
Data processing equipment
The data captured by the motion capture system needs to be corrected, processed, and combined with a 3D model to complete the computer animation. This requires us to apply data processing software or hardware to do the work. Software or hardware, they all use the computer's high-speed computing power to complete the processing of data, so that the three-dimensional model truly and naturally move. In the play, Tom Hanks wore a black tight-fitting suit with 150 sensors, so that the computer could capture his eyes, lips, eyebrows, and even the expressions and movements of each body.
Types of motion capture technology
There are many kinds of motion capture systems. Generally, they can be divided into five categories according to technical principles: mechanical, acoustic, electromagnetic, inertial sensor, optical, etc. Optical type can be divided into marked points according to different target feature types. Optical and markless dot optics. The so-called thermal energy motion capture system has recently appeared on the market, which is essentially a category of unmarked point optical motion capture, but optical imaging sensors mainly work in the near infrared or infrared range.
mechanical
Mechanical motion capture systems rely on mechanical devices to track and measure motion trajectories. A typical system consists of multiple joints and rigid links. An angle sensor is mounted in the rotatable joint to measure changes in the angle of rotation of the joint. When the device is moving, according to the angle change measured by the angle sensor and the length of the connecting rod, the position and motion trajectory of the end point of the rod in space can be obtained. X-1st is the representative of this kind of product. Its advantages are low cost, high precision and high sampling frequency. However, the biggest disadvantage is that the action performance is inconvenient. The link structure and sensor cable are very restrictive and restrictive to the performer's action. In particular, coherent movements are hindered, making it difficult to achieve true dynamic restoration.
Acoustic
Acoustic motion capture systems typically consist of a transmitting device, a receiving system, and a processing system. The transmitting device generally refers to an ultrasonic generator, and the receiving system generally consists of three or more ultrasonic probe arrays. By measuring the time or phase difference of the sound wave from a transmitting device to the sensor, the distance to the receiving sensor is determined, and the distance information obtained by the three triangular receiving sensors is used to calculate the position and direction of the ultrasonic generator to the receiver. The biggest advantage is that the cost is low, but the disadvantage is that the precision is poor, the real-time performance is not high, and it is greatly affected by factors such as noise and multiple reflection.
Electromagnetic
An electromagnetic motion capture system generally consists of a transmission source, a receiving sensor, and a data processing unit. The transmitting source generates an electromagnetic field distributed in space according to a certain time and space; the receiving sensor is placed at a key position of the performer's body, and as the performer moves in the electromagnetic field, the receiving sensor transmits the received signal to the processing through a cable or wirelessly. Unit, based on these signals, can calculate the spatial position and direction of each sensor. Polhemus and Ascension are representatives of manufacturers of such products. Their biggest features are simple, robust and real-time. The disadvantages are sensitive to metal objects. Electromagnetic field distortion caused by metal objects has a great influence on accuracy and low sampling rate. It is not conducive to the capture of fast movements, and the cable-type sensor connection also forms a constraint and obstacle to the action performance, which is not conducive to the performance of complex movements.
Inertial
The inertial sensor motion capture system consists of an attitude sensor, a signal receiver, and a data processing system. The attitude sensor is fixed to each main body part of the human body, and transmits the posture signal to the data processing system through wireless transmission means such as Bluetooth to perform motion calculation. The attitude sensor integrates inertial sensor, gravity sensor, accelerometer, magnetic sensor, micro gyroscope and other elements to obtain the posture information of each part of the limb, and then combines the length information of the bone and the bone level connection relationship to calculate the spatial position of the joint point. information. Representative products include Xsens, 3D Suit, etc. The main advantages of these products are portability, simple operation, almost unlimited performance space, and easy outdoor use. However, due to technical limitations, the disadvantages are obvious. The sensor itself cannot perform absolute spatial positioning. The spatial position information obtained by integrating the limb posture information causes different degrees of integral drift, and the spatial positioning is not accurate. On the other hand, the principle itself is based on the one-leg support and ground constraint assumptions. It is impossible to solve the motion positioning of the feet from the ground; in addition, the sensor's own weight and cable connection will also form certain constraints on the action performance, and the equipment cost will increase exponentially with the increase of the number of captured objects, and some sensors will also be subject to The ferromagnetic properties of the surrounding environment affect the accuracy.
Optical
The optical motion capture system is based on the principle of computer vision [2] [3], and the task of motion capture is accomplished by multiple high-speed cameras monitoring and tracking target feature points from different angles. Theoretically, for any point in space, as long as it can be seen by both cameras at the same time, the position of the point in space at this moment can be determined. When the camera is continuously shooting at a high enough rate, the motion trajectory of the point is obtained from the sequence of images.
Such system acquisition sensors are usually optical cameras. The difference is that the target sensors are of different types. One is that no additional markers are added to the object, and joint information extracted based on two-dimensional image features or three-dimensional shape features is used as a detection target. The system can be collectively referred to as a mark-free point optical motion capture system, and the other is to attach a marker point to the object as a target sensor. Such a system is called a marker-point optical motion capture.
1. Unmarked optics
There are three principles of unmarked point optical motion capture: the first one is based on motion capture of ordinary video images, the coordinates of joint points in two-dimensional images are extracted by two-dimensional image humanoid detection, and the joints are calculated according to multi-camera visual three-dimensional measurement. The coordinates of the three-dimensional space. Due to the cumbersome general image information, this kind of calculation is usually less robust, slow, and has poor real-time performance. The joint lacks quantitative information and the calculation error is large. This type of technology is currently in the laboratory research stage; The motion capture of the infrared camera image based on the active heat source illumination separation front and rear scene information, the so-called thermal energy motion capture, the principle is similar to the first one, but after the illumination by the heat source, the image foreground and the background separation make the humanoid detection speed greatly Enhancement improves the robustness and calculation rate of 3D reconstruction, but the heat source illuminates from a fixed direction, resulting in limited motion direction of the human body during motion capture, making it difficult to perform 360-degree motion capture. For example, turning, pitching, etc. are not applicable. And the same is not able to break through the technical barriers caused by the lack of clear joint reference information; the third is the motion capture of 3D depth information, the system based on structured optical coding projection real-time acquisition of 3D depth information of objects in the field of view, according to 3D Morphology for humanoid detection, extraction of joint motion trajectory, The watch product is Microsoft's kinect sensor [5], its motion recognition is robust, the sampling rate is high, and the price is very low. Many fans try to use kinect for motion capture, the effect is not satisfactory, this is Because the application positioning of kinect is a motion recognition sensor, rather than accurate capture, there are also problems such as large joint position calculation error and cumulative deformation of hierarchical bone movement. In general, the common problem of unmarked point motion capture is that the motion capture accuracy is low, and due to the inherent limitations of the principle, the motion freedom solution is missing (such as the spin information of the bone), which causes motion deformation.
2, marking optics
Marked optical motion capture systems generally consist of optical markers (Markers), motion capture cameras, signal transmission devices, and data processing workstations. Optical motion capture systems, commonly referred to as optical motion capture systems, are commonly referred to as point-and-click motion capture systems. The Marker point is pasted on key parts of the moving object (such as the joints of the human body). The multiple motion capture cameras detect the Marker points in real time from different angles, and the data is transmitted to the data processing workstation in real time, and the spatial coordinates of the Marker points are accurately calculated according to the triangulation principle. Then calculate the 6-degree-of-freedom movement of the bone from the principle of biokinetics.
Here, according to the different marking technology, it is divided into active and passive optical motion capture systems:
(1) Active optics
The Marker point of the active optical motion capture system consists of LEDs, which are attached to the main joint parts of the human body. The LEDs are connected by cables and powered by a power supply unit attached to the surface of the human body.
The main advantage is that the high-brightness LED is used as the optical marker, and the outdoor motion capture can be performed to a certain extent. The LED is controlled by the pulse signal to brighten and darken the LED, and the LED is time-domain coded and recognized, and the recognition is robust and has high tracking. Accuracy;
weakness is:
First, the principle of LED recognition for time-series coding essentially relies on the camera to image different Marker acquisitions at different times for ID identification, which is equivalent to successive exposures for each Marker in the same action frame, destroying the motion capture. The synchronism detected by Markers leads to motion deformation, which is not conducive to the capture of fast motion;
Second, since a large portion of the camera frame rate is used to identify different Marker points in a single frame, the effective motion frame sampling rate is low, which is also disadvantageous for fast motion capture and data analysis;
Third, the LED Marker has a small viewing angle (about 120 degrees of emission angle). A capture lens is usually integrated with two cameras for close-range acquisition. This narrow baseline structure leads to low accuracy of visual 3D measurement and is Problems such as motion occlusion still inevitably lead to frequent data loss. If data loss is caused by avoiding occlusion as much as possible, it is necessary to multiply the number of motion capture lenses to make up for the problem of blocking blind spots, and the equipment cost is also multiplied;
Fourth, due to the limitations of the timing coding, the total number of Markers that the system can support is strictly limited. Under the premise of ensuring sufficient sampling rate, the number of simultaneous acquisitions should not exceed 2, and the number of Marker points is larger. The longer the time, the more severe the motion deformation.
(2) Passive optical
Passive optical motion capture system, also known as reflective optical motion capture system, its Marker point is usually a kind of high-returning retro-reflective ball, which is attached to all major joint parts of the human body, and the LED emitted by the motion-capture lens is illuminated by the reflective ball. Reflected to the camera, for Marker detection and spatial positioning.
Its main advantages are mature technology, high precision, high sampling rate, accurate motion capture, flexible and fast performance and use. Marker points can be added and arranged at low cost, and the scope of application is wide.
The main disadvantages are:
First, sensitive to capturing sunlight in the field of view, the spot formed by sunlight on the ground may be misidentified as a Marker point, causing target interference, so the system generally needs to work normally in an indoor environment;
Second, Marker point recognition is easy to make mistakes. Since the reflective Marker point does not have unique ID information, occlusion problems during motion are likely to cause target tracking errors, which leads to confusion of Marker point ID. This situation usually leads to real-time motion capture. The animation demonstration is not good, the movement is easy to be misplaced, and the data needs to be repaired by manual intervention in the post-processing process, and the workload is greatly increased. However, the new generation of technology is equipped with advanced intelligent capture technology, with strong Marker point automatic identification and error correction capabilities, which largely meets the needs of live real-time animation demonstration, and greatly reduces the workload of manual intervention. In essence, the practicality of the system is further improved.
Advantages and disadvantages of motion capture technology
advantage
The advantage of motion capture is that the performer has a wide range of activities, no restrictions on cables and mechanical devices, and is easy to use. The sampling rate is high enough to meet the needs of most sports measurements. Marker is cheap and easy to expand. More practically, it is to facilitate the realization of various cool effects in movies and games.
Disadvantage
The system is expensive, although it captures real-time motion, but post-processing (including Marker's recognition, tracking, and calculation of spatial coordinates) takes a long time. Such systems are sensitive to the lighting and reflection of the performance venue. Device calibration is also cumbersome, especially when the movement is complicated. Marker in different parts is easy to confuse, occlude, and produce erroneous results, often requiring manual intervention and post-processing. Due to various limitations, almost all optical tracking systems rely on post-processing programs to analyze, process, and organize captured data before applying it to an animated character model.
Main application areas of motion capture technology
Animation
Using motion capture technology for animation can greatly improve the level of animation. It greatly improves the efficiency of animation production, reduces costs, and makes the animation process more intuitive and more vivid.
Virtual reality system
In order to realize the interaction between the human and the virtual environment and the system, it is necessary to determine the position and direction of the participant's head, hands, body, etc., accurately track the movements of the measurement participants, and detect these actions in real time in order to feed back the data to Display and control system. These tasks are indispensable for virtual reality systems, which is the research content of motion capture technology.
Robot remote control
The robot transmits the information of the dangerous environment to the controller, and the controller makes various actions according to the information. The motion capture system captures the motion, transmits it to the robot in real time and controls it to perform the same action. Compared with the traditional remote control method, this system can realize more intuitive, detailed, complex, flexible and fast motion control, greatly improving the ability of the robot to cope with complex situations. This technology is of particular importance in the current situation where the robot's full autonomous control is not yet mature.
Interactive game
Motion capture technology can be used to capture various actions of the player to drive the action of the characters in the game environment, giving the player a new sense of participation and enhancing the realism and interactivity of the game.
Sports training
Motion capture technology can capture the movements of athletes, facilitate quantitative analysis, combine human physiology, physics, research and improvement methods, so that sports training can get rid of purely relying on experience and enter the era of theoretical and digital. It is also possible to capture the movements of athletes with poor grades and compare them with the movements of elite athletes to help them train.
In addition, in the fields of ergonomics research, simulation training, biomechanical research, etc., motion capture technology is also very promising. It can be expected that with the development of technology itself and the improvement of the technical level of related application fields, motion capture technology will be more and more widely used.
This article is an original article on the sensor technology platform. Reprinted need to contact us for authorization! We will make a complaint without permission to reprint!
ShenZhen Haofa Metal Precision Parts Technology Co., Ltd. , https://www.haofametal.com