OpenShot Library | libopenshot  0.2.5
Timeline.h
Go to the documentation of this file.
1 /**
2  * @file
3  * @brief Header file for Timeline class
4  * @author Jonathan Thomas <jonathan@openshot.org>
5  *
6  * @ref License
7  */
8 
9 /* LICENSE
10  *
11  * Copyright (c) 2008-2019 OpenShot Studios, LLC
12  * <http://www.openshotstudios.com/>. This file is part of
13  * OpenShot Library (libopenshot), an open-source project dedicated to
14  * delivering high quality video editing and animation solutions to the
15  * world. For more information visit <http://www.openshot.org/>.
16  *
17  * OpenShot Library (libopenshot) is free software: you can redistribute it
18  * and/or modify it under the terms of the GNU Lesser General Public License
19  * as published by the Free Software Foundation, either version 3 of the
20  * License, or (at your option) any later version.
21  *
22  * OpenShot Library (libopenshot) is distributed in the hope that it will be
23  * useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
24  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
25  * GNU Lesser General Public License for more details.
26  *
27  * You should have received a copy of the GNU Lesser General Public License
28  * along with OpenShot Library. If not, see <http://www.gnu.org/licenses/>.
29  */
30 
31 #ifndef OPENSHOT_TIMELINE_H
32 #define OPENSHOT_TIMELINE_H
33 
34 #include <list>
35 #include <memory>
36 #include <set>
37 #include <QtGui/QImage>
38 #include <QtGui/QPainter>
39 #include "CacheBase.h"
40 #include "CacheDisk.h"
41 #include "CacheMemory.h"
42 #include "Color.h"
43 #include "Clip.h"
44 #include "CrashHandler.h"
45 #include "Point.h"
46 #include "EffectBase.h"
47 #include "Effects.h"
48 #include "EffectInfo.h"
49 #include "Fraction.h"
50 #include "Frame.h"
51 #include "FrameMapper.h"
52 #include "KeyFrame.h"
53 #include "OpenMPUtilities.h"
54 #include "ReaderBase.h"
55 #include "Settings.h"
56 
57 namespace openshot {
58 
59  /// Comparison method for sorting clip pointers (by Layer and then Position). Clips are sorted
60  /// from lowest layer to top layer (since that is the sequence they need to be combined), and then
61  /// by position (left to right).
62  struct CompareClips{
63  bool operator()( Clip* lhs, Clip* rhs){
64  if( lhs->Layer() < rhs->Layer() ) return true;
65  if( lhs->Layer() == rhs->Layer() && lhs->Position() <= rhs->Position() ) return true;
66  return false;
67  }};
68 
69  /// Comparison method for sorting effect pointers (by Position, Layer, and Order). Effects are sorted
70  /// from lowest layer to top layer (since that is sequence clips are combined), and then by
71  /// position, and then by effect order.
73  bool operator()( EffectBase* lhs, EffectBase* rhs){
74  if( lhs->Layer() < rhs->Layer() ) return true;
75  if( lhs->Layer() == rhs->Layer() && lhs->Position() < rhs->Position() ) return true;
76  if( lhs->Layer() == rhs->Layer() && lhs->Position() == rhs->Position() && lhs->Order() > rhs->Order() ) return true;
77  return false;
78  }};
79 
80  /**
81  * @brief This class represents a timeline
82  *
83  * The timeline is one of the <b>most important</b> features of a video editor, and controls all
84  * aspects of how video, image, and audio clips are combined together, and how the final
85  * video output will be rendered. It has a collection of layers and clips, that arrange,
86  * sequence, and generate the final video output.
87  *
88  * The <b>following graphic</b> displays a timeline, and how clips can be arranged, scaled, and layered together. It
89  * also demonstrates how the viewport can be scaled smaller than the canvas, which can be used to zoom and pan around the
90  * canvas (i.e. pan & scan).
91  * \image html /doc/images/Timeline_Layers.png
92  *
93  * The <b>following graphic</b> displays how the playhead determines which frames to combine and layer.
94  * \image html /doc/images/Playhead.png
95  *
96  * Lets take a look at what the code looks like:
97  * @code
98  * // Create a Timeline
99  * Timeline t(1280, // width
100  * 720, // height
101  * Fraction(25,1), // framerate
102  * 44100, // sample rate
103  * 2 // channels
104  * ChannelLayout::LAYOUT_STEREO,
105  * );
106  *
107  * // Create some clips
108  * Clip c1(new ImageReader("MyAwesomeLogo.jpeg"));
109  * Clip c2(new FFmpegReader("BackgroundVideo.webm"));
110  *
111  * // CLIP 1 (logo) - Set some clip properties (with Keyframes)
112  * c1.Position(0.0); // Set the position or location (in seconds) on the timeline
113  * c1.gravity = GRAVITY_LEFT; // Set the alignment / gravity of the clip (position on the screen)
114  * c1.scale = SCALE_CROP; // Set the scale mode (how the image is resized to fill the screen)
115  * c1.Layer(1); // Set the layer of the timeline (higher layers cover up images of lower layers)
116  * c1.Start(0.0); // Set the starting position of the video (trim the left side of the video)
117  * c1.End(16.0); // Set the ending position of the video (trim the right side of the video)
118  * c1.alpha.AddPoint(1, 0.0); // Set the alpha to transparent on frame #1
119  * c1.alpha.AddPoint(500, 0.0); // Keep the alpha transparent until frame #500
120  * c1.alpha.AddPoint(565, 1.0); // Animate the alpha from transparent to visible (between frame #501 and #565)
121  *
122  * // CLIP 2 (background video) - Set some clip properties (with Keyframes)
123  * c2.Position(0.0); // Set the position or location (in seconds) on the timeline
124  * c2.Start(10.0); // Set the starting position of the video (trim the left side of the video)
125  * c2.Layer(0); // Set the layer of the timeline (higher layers cover up images of lower layers)
126  * c2.alpha.AddPoint(1, 1.0); // Set the alpha to visible on frame #1
127  * c2.alpha.AddPoint(150, 0.0); // Animate the alpha to transparent (between frame 2 and frame #150)
128  * c2.alpha.AddPoint(360, 0.0, LINEAR); // Keep the alpha transparent until frame #360
129  * c2.alpha.AddPoint(384, 1.0); // Animate the alpha to visible (between frame #360 and frame #384)
130  *
131  * // Add clips to timeline
132  * t.AddClip(&c1);
133  * t.AddClip(&c2);
134  *
135  * // Open the timeline reader
136  * t.Open();
137  *
138  * // Get frame number 1 from the timeline (This will generate a new frame, made up from the previous clips and settings)
139  * std::shared_ptr<Frame> f = t.GetFrame(1);
140  *
141  * // Now that we have an openshot::Frame object, lets have some fun!
142  * f->Display(); // Display the frame on the screen
143  *
144  * // Close the timeline reader
145  * t.Close();
146  * @endcode
147  */
148  class Timeline : public ReaderBase {
149  private:
150  bool is_open; ///<Is Timeline Open?
151  bool auto_map_clips; ///< Auto map framerates and sample rates to all clips
152  std::list<Clip*> clips; ///<List of clips on this timeline
153  std::list<Clip*> closing_clips; ///<List of clips that need to be closed
154  std::map<Clip*, Clip*> open_clips; ///<List of 'opened' clips on this timeline
155  std::list<EffectBase*> effects; ///<List of clips on this timeline
156  CacheBase *final_cache; ///<Final cache of timeline frames
157  std::set<FrameMapper*> allocated_frame_mappers; ///< all the frame mappers we allocated and must free
158  bool managed_cache; ///< Does this timeline instance manage the cache object
159 
160  /// Process a new layer of video or audio
161  void add_layer(std::shared_ptr<Frame> new_frame, Clip* source_clip, int64_t clip_frame_number, int64_t timeline_frame_number, bool is_top_clip, float max_volume);
162 
163  /// Apply a FrameMapper to a clip which matches the settings of this timeline
164  void apply_mapper_to_clip(Clip* clip);
165 
166  /// Apply JSON Diffs to various objects contained in this timeline
167  void apply_json_to_clips(Json::Value change); ///<Apply JSON diff to clips
168  void apply_json_to_effects(Json::Value change); ///< Apply JSON diff to effects
169  void apply_json_to_effects(Json::Value change, EffectBase* existing_effect); ///<Apply JSON diff to a specific effect
170  void apply_json_to_timeline(Json::Value change); ///<Apply JSON diff to timeline properties
171 
172  /// Calculate time of a frame number, based on a framerate
173  double calculate_time(int64_t number, Fraction rate);
174 
175  /// Find intersecting (or non-intersecting) openshot::Clip objects
176  ///
177  /// @returns A list of openshot::Clip objects
178  /// @param requested_frame The frame number that is requested.
179  /// @param number_of_frames The number of frames to check
180  /// @param include Include or Exclude intersecting clips
181  std::vector<Clip*> find_intersecting_clips(int64_t requested_frame, int number_of_frames, bool include);
182 
183  /// Get or generate a blank frame
184  std::shared_ptr<Frame> GetOrCreateFrame(Clip* clip, int64_t number);
185 
186  /// Apply effects to the source frame (if any)
187  std::shared_ptr<Frame> apply_effects(std::shared_ptr<Frame> frame, int64_t timeline_frame_number, int layer);
188 
189  /// Compare 2 floating point numbers for equality
190  bool isEqual(double a, double b);
191 
192  /// Sort clips by position on the timeline
193  void sort_clips();
194 
195  /// Sort effects by position on the timeline
196  void sort_effects();
197 
198  /// Update the list of 'opened' clips
199  void update_open_clips(Clip *clip, bool does_clip_intersect);
200 
201  public:
202 
203  /// @brief Default Constructor for the timeline (which sets the canvas width and height and FPS)
204  /// @param width The width of the timeline (and thus, the generated openshot::Frame objects)
205  /// @param height The height of the timeline (and thus, the generated openshot::Frame objects)
206  /// @param fps The frames rate of the timeline
207  /// @param sample_rate The sample rate of the timeline's audio
208  /// @param channels The number of audio channels of the timeline
209  /// @param channel_layout The channel layout (i.e. mono, stereo, 3 point surround, etc...)
210  Timeline(int width, int height, Fraction fps, int sample_rate, int channels, ChannelLayout channel_layout);
211 
212  virtual ~Timeline();
213 
214  /// @brief Add an openshot::Clip to the timeline
215  /// @param clip Add an openshot::Clip to the timeline. A clip can contain any type of Reader.
216  void AddClip(Clip* clip);
217 
218  /// @brief Add an effect to the timeline
219  /// @param effect Add an effect to the timeline. An effect can modify the audio or video of an openshot::Frame.
220  void AddEffect(EffectBase* effect);
221 
222  /// Apply the timeline's framerate and samplerate to all clips
223  void ApplyMapperToClips();
224 
225  /// Determine if clips are automatically mapped to the timeline's framerate and samplerate
226  bool AutoMapClips() { return auto_map_clips; };
227 
228  /// @brief Automatically map all clips to the timeline's framerate and samplerate
229  void AutoMapClips(bool auto_map) { auto_map_clips = auto_map; };
230 
231  /// Clear all cache for this timeline instance, and all clips, mappers, and readers under it
232  void ClearAllCache();
233 
234  /// Return a list of clips on the timeline
235  std::list<Clip*> Clips() { return clips; };
236 
237  /// Close the timeline reader (and any resources it was consuming)
238  void Close();
239 
240  /// Return the list of effects on the timeline
241  std::list<EffectBase*> Effects() { return effects; };
242 
243  /// Get the cache object used by this reader
244  CacheBase* GetCache() { return final_cache; };
245 
246  /// Set the cache object used by this reader. You must now manage the lifecycle
247  /// of this cache object though (Timeline will not delete it for you).
248  void SetCache(CacheBase* new_cache);
249 
250  /// Get an openshot::Frame object for a specific frame number of this timeline.
251  ///
252  /// @returns The requested frame (containing the image)
253  /// @param requested_frame The frame number that is requested.
254  std::shared_ptr<Frame> GetFrame(int64_t requested_frame);
255 
256  // Curves for the viewport
257  Keyframe viewport_scale; ///<Curve representing the scale of the viewport (0 to 100)
258  Keyframe viewport_x; ///<Curve representing the x coordinate for the viewport
259  Keyframe viewport_y; ///<Curve representing the y coordinate for the viewport
260 
261  // Background color
262  Color color; ///<Background color of timeline canvas
263 
264  /// Determine if reader is open or closed
265  bool IsOpen() { return is_open; };
266 
267  /// Return the type name of the class
268  std::string Name() { return "Timeline"; };
269 
270  /// Get and Set JSON methods
271  std::string Json() const override; ///< Generate JSON string of this object
272  void SetJson(const std::string value); ///< Load JSON string into this object
273  Json::Value JsonValue() const override; ///< Generate Json::Value for this object
274  void SetJsonValue(const Json::Value root); ///< Load Json::Value into this object
275 
276  /// Set Max Image Size (used for performance optimization). Convenience function for setting
277  /// Settings::Instance()->MAX_WIDTH and Settings::Instance()->MAX_HEIGHT.
278  void SetMaxSize(int width, int height);
279 
280  /// @brief Apply a special formatted JSON object, which represents a change to the timeline (add, update, delete)
281  /// This is primarily designed to keep the timeline (and its child objects... such as clips and effects) in sync
282  /// with another application... such as OpenShot Video Editor (http://www.openshot.org).
283  /// @param value A JSON string containing a key, value, and type of change.
284  void ApplyJsonDiff(std::string value);
285 
286  /// Open the reader (and start consuming resources)
287  void Open();
288 
289  /// @brief Remove an openshot::Clip from the timeline
290  /// @param clip Remove an openshot::Clip from the timeline.
291  void RemoveClip(Clip* clip);
292 
293  /// @brief Remove an effect from the timeline
294  /// @param effect Remove an effect from the timeline.
295  void RemoveEffect(EffectBase* effect);
296  };
297 
298 
299 }
300 
301 #endif
CacheBase * GetCache()
Get the cache object used by this reader.
Definition: Timeline.h:244
Header file for Fraction class.
std::list< Clip * > Clips()
Return a list of clips on the timeline.
Definition: Timeline.h:235
This abstract class is the base class, used by all effects in libopenshot.
Definition: EffectBase.h:66
Keyframe viewport_scale
Curve representing the scale of the viewport (0 to 100)
Definition: Timeline.h:257
Header file for ReaderBase class.
bool AutoMapClips()
Determine if clips are automatically mapped to the timeline&#39;s framerate and samplerate.
Definition: Timeline.h:226
Header file for OpenMPUtilities (set some common macros)
Header file for Point class.
This header includes all commonly used effects for libopenshot, for ease-of-use.
Keyframe viewport_y
Curve representing the y coordinate for the viewport.
Definition: Timeline.h:259
This abstract class is the base class, used by all readers in libopenshot.
Definition: ReaderBase.h:97
Header file for the Keyframe class.
Header file for CacheMemory class.
Header file for CacheBase class.
This class represents a clip (used to arrange readers on the timeline)
Definition: Clip.h:95
Header file for Frame class.
Header file for Clip class.
bool operator()(Clip *lhs, Clip *rhs)
Definition: Timeline.h:63
This class represents a fraction.
Definition: Fraction.h:45
Header file for the FrameMapper class.
std::list< EffectBase * > Effects()
Return the list of effects on the timeline.
Definition: Timeline.h:241
All cache managers in libopenshot are based on this CacheBase class.
Definition: CacheBase.h:49
ChannelLayout
This enumeration determines the audio channel layout (such as stereo, mono, 5 point surround...
Header file for global Settings class.
int Order() const
Get the order that this effect should be executed.
Definition: EffectBase.h:104
Header file for Color class.
void AutoMapClips(bool auto_map)
Automatically map all clips to the timeline&#39;s framerate and samplerate.
Definition: Timeline.h:229
bool IsOpen()
Determine if reader is open or closed.
Definition: Timeline.h:265
This class represents a color (used on the timeline and clips)
Definition: Color.h:45
This namespace is the default namespace for all code in the openshot library.
Header file for EffectBase class.
std::string Name()
Return the type name of the class.
Definition: Timeline.h:268
Keyframe viewport_x
Curve representing the x coordinate for the viewport.
Definition: Timeline.h:258
Header file for CacheDisk class.
Header file for CrashHandler class.
Color color
Background color of timeline canvas.
Definition: Timeline.h:262
float Position() const
Get position on timeline (in seconds)
Definition: ClipBase.h:77
Header file for the EffectInfo class.
A Keyframe is a collection of Point instances, which is used to vary a number or property over time...
Definition: KeyFrame.h:64
int Layer() const
Get layer of clip on timeline (lower number is covered by higher numbers)
Definition: ClipBase.h:78
bool operator()(EffectBase *lhs, EffectBase *rhs)
Definition: Timeline.h:73
This class represents a timeline.
Definition: Timeline.h:148