1How to Use Background Subtraction Methods {#tutorial_background_subtraction}
2=========================================
3
4-   Background subtraction (BS) is a common and widely used technique for generating a foreground
5    mask (namely, a binary image containing the pixels belonging to moving objects in the scene) by
6    using static cameras.
7-   As the name suggests, BS calculates the foreground mask performing a subtraction between the
8    current frame and a background model, containing the static part of the scene or, more in
9    general, everything that can be considered as background given the characteristics of the
10    observed scene.
11
12    ![](images/Background_Subtraction_Tutorial_Scheme.png)
13
14-   Background modeling consists of two main steps:
15
16    -#  Background Initialization;
17    -#  Background Update.
18
19    In the first step, an initial model of the background is computed, while in the second step that
20    model is updated in order to adapt to possible changes in the scene.
21
22-   In this tutorial we will learn how to perform BS by using OpenCV. As input, we will use data
23    coming from the publicly available data set [Background Models Challenge
24    (BMC)](http://bmc.univ-bpclermont.fr/) .
25
26Goals
27-----
28
29In this tutorial you will learn how to:
30
31-#  Read data from videos by using @ref cv::VideoCapture or image sequences by using @ref
32    cv::imread ;
33-#  Create and update the background model by using @ref cv::BackgroundSubtractor class;
34-#  Get and show the foreground mask by using @ref cv::imshow ;
35-#  Save the output by using @ref cv::imwrite to quantitatively evaluate the results.
36
37Code
38----
39
40In the following you can find the source code. We will let the user chose to process either a video
41file or a sequence of images.
42
43Two different methods are used to generate two foreground masks:
44-#  cv::bgsegm::BackgroundSubtractorMOG
45-#  @ref cv::BackgroundSubtractorMOG2
46
47The results as well as the input data are shown on the screen.
48The source file can be downloaded [here ](https://github.com/Itseez/opencv/tree/master/samples/cpp/tutorial_code/video/bg_sub.cpp).
49
50@include samples/cpp/tutorial_code/video/bg_sub.cpp
51
52Explanation
53-----------
54
55We discuss the main parts of the above code:
56
57-#  First, three Mat objects are allocated to store the current frame and two foreground masks,
58    obtained by using two different BS algorithms.
59    @code{.cpp}
60    Mat frame; //current frame
61    Mat fgMaskMOG; //fg mask generated by MOG method
62    Mat fgMaskMOG2; //fg mask fg mask generated by MOG2 method
63    @endcode
64-#  Two @ref cv::BackgroundSubtractor objects will be used to generate the foreground masks. In this
65    example, default parameters are used, but it is also possible to declare specific parameters in
66    the create function.
67    @code{.cpp}
68    Ptr<BackgroundSubtractor> pMOG; //MOG Background subtractor
69    Ptr<BackgroundSubtractor> pMOG2; //MOG2 Background subtractor
70    ...
71    //create Background Subtractor objects
72    pMOG = createBackgroundSubtractorMOG(); //MOG approach
73    pMOG2 = createBackgroundSubtractorMOG2(); //MOG2 approach
74    @endcode
75-#  The command line arguments are analysed. The user can chose between two options:
76    -   video files (by choosing the option -vid);
77    -   image sequences (by choosing the option -img).
78    @code{.cpp}
79    if(strcmp(argv[1], "-vid") == 0) {
80      //input data coming from a video
81      processVideo(argv[2]);
82    }
83    else if(strcmp(argv[1], "-img") == 0) {
84      //input data coming from a sequence of images
85      processImages(argv[2]);
86    }
87    @endcode
88-#  Suppose you want to process a video file. The video is read until the end is reached or the user
89    presses the button 'q' or the button 'ESC'.
90    @code{.cpp}
91    while( (char)keyboard != 'q' && (char)keyboard != 27 ){
92      //read the current frame
93      if(!capture.read(frame)) {
94        cerr << "Unable to read next frame." << endl;
95        cerr << "Exiting..." << endl;
96        exit(EXIT_FAILURE);
97      }
98    @endcode
99-#  Every frame is used both for calculating the foreground mask and for updating the background. If
100    you want to change the learning rate used for updating the background model, it is possible to
101    set a specific learning rate by passing a third parameter to the 'apply' method.
102    @code{.cpp}
103    //update the background model
104    pMOG->apply(frame, fgMaskMOG);
105    pMOG2->apply(frame, fgMaskMOG2);
106    @endcode
107-#  The current frame number can be extracted from the @ref cv::VideoCapture object and stamped in
108    the top left corner of the current frame. A white rectangle is used to highlight the black
109    colored frame number.
110    @code{.cpp}
111    //get the frame number and write it on the current frame
112    stringstream ss;
113    rectangle(frame, cv::Point(10, 2), cv::Point(100,20),
114              cv::Scalar(255,255,255), -1);
115    ss << capture.get(CAP_PROP_POS_FRAMES);
116    string frameNumberString = ss.str();
117    putText(frame, frameNumberString.c_str(), cv::Point(15, 15),
118            FONT_HERSHEY_SIMPLEX, 0.5 , cv::Scalar(0,0,0));
119    @endcode
120-#  We are ready to show the current input frame and the results.
121    @code{.cpp}
122    //show the current frame and the fg masks
123    imshow("Frame", frame);
124    imshow("FG Mask MOG", fgMaskMOG);
125    imshow("FG Mask MOG 2", fgMaskMOG2);
126    @endcode
127-#  The same operations listed above can be performed using a sequence of images as input. The
128    processImage function is called and, instead of using a @ref cv::VideoCapture object, the images
129    are read by using @ref cv::imread , after individuating the correct path for the next frame to
130    read.
131    @code{.cpp}
132    //read the first file of the sequence
133    frame = imread(fistFrameFilename);
134    if(!frame.data){
135      //error in opening the first image
136      cerr << "Unable to open first image frame: " << fistFrameFilename << endl;
137      exit(EXIT_FAILURE);
138    }
139    ...
140    //search for the next image in the sequence
141    ostringstream oss;
142    oss << (frameNumber + 1);
143    string nextFrameNumberString = oss.str();
144    string nextFrameFilename = prefix + nextFrameNumberString + suffix;
145    //read the next frame
146    frame = imread(nextFrameFilename);
147    if(!frame.data){
148      //error in opening the next image in the sequence
149      cerr << "Unable to open image frame: " << nextFrameFilename << endl;
150      exit(EXIT_FAILURE);
151    }
152    //update the path of the current frame
153    fn.assign(nextFrameFilename);
154    @endcode
155    Note that this example works only on image sequences in which the filename format is \<n\>.png,
156    where n is the frame number (e.g., 7.png).
157
158Results
159-------
160
161-   Given the following input parameters:
162    @code{.cpp}
163    -vid Video_001.avi
164    @endcode
165    The output of the program will look as the following:
166
167    ![](images/Background_Subtraction_Tutorial_Result_1.png)
168
169-   The video file Video_001.avi is part of the [Background Models Challenge
170    (BMC)](http://bmc.univ-bpclermont.fr/) data set and it can be downloaded from the following link
171    [Video_001](http://bmc.univ-bpclermont.fr/sites/default/files/videos/evaluation/Video_001.zip)
172    (about 32 MB).
173-   If you want to process a sequence of images, then the '-img' option has to be chosen:
174    @code{.cpp}
175    -img 111_png/input/1.png
176    @endcode
177    The output of the program will look as the following:
178
179    ![](images/Background_Subtraction_Tutorial_Result_2.png)
180
181-   The sequence of images used in this example is part of the [Background Models Challenge
182    (BMC)](http://bmc.univ-bpclermont.fr/) dataset and it can be downloaded from the following link
183    [sequence 111](http://bmc.univ-bpclermont.fr/sites/default/files/videos/learning/111_png.zip)
184    (about 708 MB). Please, note that this example works only on sequences in which the filename
185    format is \<n\>.png, where n is the frame number (e.g., 7.png).
186
187Evaluation
188----------
189
190To quantitatively evaluate the results obtained, we need to:
191
192-   Save the output images;
193-   Have the ground truth images for the chosen sequence.
194
195In order to save the output images, we can use @ref cv::imwrite . Adding the following code allows
196for saving the foreground masks.
197@code{.cpp}
198string imageToSave = "output_MOG_" + frameNumberString + ".png";
199bool saved = imwrite(imageToSave, fgMaskMOG);
200if(!saved) {
201  cerr << "Unable to save " << imageToSave << endl;
202}
203@endcode
204Once we have collected the result images, we can compare them with the ground truth data. There
205exist several publicly available sequences for background subtraction that come with ground truth
206data. If you decide to use the [Background Models Challenge (BMC)](http://bmc.univ-bpclermont.fr/),
207then the result images can be used as input for the [BMC
208Wizard](http://bmc.univ-bpclermont.fr/?q=node/7). The wizard can compute different measures about
209the accuracy of the results.
210
211References
212----------
213
214-   [Background Models Challenge (BMC) website](http://bmc.univ-bpclermont.fr/)
215-   A Benchmark Dataset for Foreground/Background Extraction @cite vacavant2013benchmark
216