Aegisub/src/audio_karaoke.h

166 lines
6.7 KiB
C
Raw Normal View History

// Copyright (c) 2013, Thomas Goyne <plorkyeran@aegisub.org>
2006-01-16 22:02:54 +01:00
//
// Permission to use, copy, modify, and distribute this software for any
// purpose with or without fee is hereby granted, provided that the above
// copyright notice and this permission notice appear in all copies.
2006-01-16 22:02:54 +01:00
//
// THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
// WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
// ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
// WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
// ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
// OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
2006-01-16 22:02:54 +01:00
//
// Aegisub Project http://www.aegisub.org/
#include <libaegisub/signal.h>
2013-06-10 15:58:13 +02:00
#include <memory>
#include <set>
#include <unordered_map>
#include <vector>
#include <wx/bitmap.h>
2012-10-06 18:39:44 +02:00
#include <wx/timer.h>
#include <wx/window.h>
2006-01-16 22:02:54 +01:00
class AssDialogue;
class AssKaraoke;
class wxButton;
namespace agi { class AudioProvider; }
namespace agi { struct Context; }
/// @class AudioKaraoke
/// @brief Syllable split and join UI for karaoke
///
/// This class has two main responsibilities: the syllable split/join UI, and
/// the karaoke mode controller. The split/join UI consists of the dialogue
/// line with spaces and lines at each syllable split point. Clicking on a line
/// removes that \k tag; clicking anywhere else inserts a new \k tag with
/// interpolated duration. Added or removed splits are not autocommitted and
/// must be explicitly accepted or rejected. This is for two reasons:
/// 1. It's easy for a stray click on the split/join bar to go unnoticed,
/// making autocommitting somewhat error-prone.
/// 2. When a line with zero \k tags is activated, it's automatically split
/// at each space. This clearly should not automatically update the line
/// (changing the active selection should never directly change the file
/// itself), so there must be a notion of pending splits.
///
/// As the karaoke controller, it owns the AssKaraoke instance shared by this
/// class and the karaoke timing controller, and is responsible for switching
/// between timing controllers when entering and leaving karaoke mode. Ideally
/// the creation of the dialogue timing controller should probably be done
/// elsewhere, but there currently isn't any particularly appropriate place and
/// it's not worth caring about. The KaraokeController duties should perhaps be
/// split off into its own class, but at the moment they're insignificant
/// enough that it's not worth it.
///
/// The shared AssKaraoke instance is primarily to improve the handling of
/// pending splits. When a split is added removed, or a line is autosplit,
/// the audio display immediately reflects the changes, but the file is not
/// actually updated until the line is committed (which if auto-commit timing
/// changes is on, will happen as soon as the user adjusts the timing of the
/// new syllable).
class AudioKaraoke final : public wxWindow {
agi::Context *c; ///< Project context
agi::signal::Connection file_changed; ///< File changed slot
agi::signal::Connection audio_opened; ///< Audio opened connection
agi::signal::Connection audio_closed; ///< Audio closed connection
agi::signal::Connection active_line_changed;
Audio/Timing: implement tap-to-time Tap-to-time provides the user the ability to tap to the lyrics/syllables of the song in order to time lines or karaoke. It consists of these extra UI interactions: - **Indicator**: tap marker: a designated marker that can be moved to the current audio position; indicated in: - the audio display by a green arrow underneath a marker - the karaoke display by a green-colored syllable - **Control**: tap marker: the tap marker can be changed by selecting syllables on audio display in karaoke mode, or clicking the markers on audio display in dialogue mode - **Control**: ctrl-right-click audio display: starts playing the audio from that exact position until the end of the file - **Option**: Timing/Tap To Time: enables the tap marker indicator and commands - **Button**: time_opt_tap_to_time: toggles the Timing/Tap To Time option - **Button**: time_tap_connect (hotkey I): a command that: - moves the tap marker's position to the current playing audio position - sets the next marker to be the tap marker - if the tap marker is already the last marker AND BOTH autocommit AND next-line-on-commit is ON, will move onto the next line - if moved on to the next line, also sets the start marker to the current audio position, so the two lines are connected, and moves to the next tap marker (essentially reinvoking time_tap_connect once) - **Button**: time_tap_no_connect (hotkey O): similar to time_tap_connect, except it will not set the next line's start position even if moved to the next line Expected workflow: 1) User loads song lyrics 2) User splits each line into syllables 3) User turns on tap-to-time, autocommit, and next-line-on-commit 4) User plays audio from beginning, tapping time_tap_connect to each syllable, occasionally tapping time_tap_no_connect when a break between lines is desired 5) If user messes up a line, they can set the tap marker to where they want to restart from, and ctrl-right-click to start the audio a few seconds before it 6) Syllables can be split/merged at will, and adjustments to timing can be done using normal karaoke timing controls
2018-10-21 09:42:33 +02:00
agi::signal::Connection tap_to_time_toggled;
/// Currently active dialogue line
2013-12-12 01:29:48 +01:00
AssDialogue *active_line = nullptr;
/// Karaoke data
2013-06-10 15:58:13 +02:00
std::unique_ptr<AssKaraoke> kara;
/// Current line's stripped text with spaces added between each syllable
std::vector<wxString> spaced_text;
2006-01-16 22:02:54 +01:00
/// spaced_text + syl_lines rendered to a bitmap
wxBitmap rendered_line;
/// Indexes in spaced_text which are the beginning of syllables
std::vector<int> syl_start_points;
/// x coordinate in pixels of the separator lines of each syllable
std::vector<int> syl_lines;
/// Left x coordinate of each character in spaced_text in pixels
std::vector<int> char_x;
/// Mapping from character index to byte position in the relevant syllable's text
std::vector<size_t> char_to_byte;
/// Cached width of characters from GetTextExtent
std::unordered_map<std::string, int> char_widths;
2013-12-12 01:29:48 +01:00
int scroll_x = 0; ///< Distance the display has been shifted to the left in pixels
int scroll_dir = 0; ///< Direction the display will be scrolled on scroll_timer ticks (+/- 1)
wxTimer scroll_timer; ///< Timer to scroll every 50ms when user holds down scroll button
2013-12-12 01:29:48 +01:00
int char_height = 0; ///< Maximum character height in pixels
int char_width = 0; ///< Maximum character width in pixels
int mouse_pos = 0; ///< Last x coordinate of the mouse
bool click_will_remove_split = false; ///< If true a click at mouse_pos will remove a split rather than adding one
wxFont split_font; ///< Font used in the split/join interface
size_t tap_syl_start = 0; ///< Tap-to-time syllable start character index
size_t tap_syl_end = 0; ///< Tap-to-time syllable end character index
Audio/Timing: implement tap-to-time Tap-to-time provides the user the ability to tap to the lyrics/syllables of the song in order to time lines or karaoke. It consists of these extra UI interactions: - **Indicator**: tap marker: a designated marker that can be moved to the current audio position; indicated in: - the audio display by a green arrow underneath a marker - the karaoke display by a green-colored syllable - **Control**: tap marker: the tap marker can be changed by selecting syllables on audio display in karaoke mode, or clicking the markers on audio display in dialogue mode - **Control**: ctrl-right-click audio display: starts playing the audio from that exact position until the end of the file - **Option**: Timing/Tap To Time: enables the tap marker indicator and commands - **Button**: time_opt_tap_to_time: toggles the Timing/Tap To Time option - **Button**: time_tap_connect (hotkey I): a command that: - moves the tap marker's position to the current playing audio position - sets the next marker to be the tap marker - if the tap marker is already the last marker AND BOTH autocommit AND next-line-on-commit is ON, will move onto the next line - if moved on to the next line, also sets the start marker to the current audio position, so the two lines are connected, and moves to the next tap marker (essentially reinvoking time_tap_connect once) - **Button**: time_tap_no_connect (hotkey O): similar to time_tap_connect, except it will not set the next line's start position even if moved to the next line Expected workflow: 1) User loads song lyrics 2) User splits each line into syllables 3) User turns on tap-to-time, autocommit, and next-line-on-commit 4) User plays audio from beginning, tapping time_tap_connect to each syllable, occasionally tapping time_tap_no_connect when a break between lines is desired 5) If user messes up a line, they can set the tap marker to where they want to restart from, and ctrl-right-click to start the audio a few seconds before it 6) Syllables can be split/merged at will, and adjustments to timing can be done using normal karaoke timing controls
2018-10-21 09:42:33 +02:00
2013-12-12 01:29:48 +01:00
bool enabled = false; ///< Is karaoke mode enabled?
2006-01-16 22:02:54 +01:00
wxButton *accept_button; ///< Accept pending splits button
wxButton *cancel_button; ///< Revert pending changes
wxWindow *split_area; ///< The split/join window
2006-01-16 22:02:54 +01:00
/// Load syllable data from the currently active line
void LoadFromLine();
/// Cache presentational data from the loaded syllable data
void SetDisplayText();
2006-01-16 22:02:54 +01:00
/// Helper function for context menu creation
2012-12-30 00:53:56 +01:00
void AddMenuItem(wxMenu &menu, std::string const& tag, wxString const& help, std::string const& selected);
/// Set the karaoke tags for the selected syllables to the indicated one
2012-12-30 00:53:56 +01:00
void SetTagType(std::string const& new_type);
2006-01-16 22:02:54 +01:00
/// Prerender the current line along with syllable split lines
void RenderText();
/// Refresh the area of the display around a single character
/// @param pos Index in spaced_text
void LimitedRefresh(int pos);
2006-01-16 22:02:54 +01:00
/// Reset all pending split information and return to normal mode
void CancelSplit();
/// Apply any pending split information to the syllable data and return to normal mode
void AcceptSplit();
void OnActiveLineChanged(AssDialogue *new_line);
void OnContextMenu(wxContextMenuEvent&);
void OnEnableButton(wxCommandEvent &evt);
void OnFileChanged(int type, const AssDialogue *changed);
void OnMouse(wxMouseEvent &event);
void OnPaint(wxPaintEvent &event);
void OnSize(wxSizeEvent &event);
void OnAudioOpened(agi::AudioProvider *provider);
void OnScrollTimer(wxTimerEvent &event);
Audio/Timing: implement tap-to-time Tap-to-time provides the user the ability to tap to the lyrics/syllables of the song in order to time lines or karaoke. It consists of these extra UI interactions: - **Indicator**: tap marker: a designated marker that can be moved to the current audio position; indicated in: - the audio display by a green arrow underneath a marker - the karaoke display by a green-colored syllable - **Control**: tap marker: the tap marker can be changed by selecting syllables on audio display in karaoke mode, or clicking the markers on audio display in dialogue mode - **Control**: ctrl-right-click audio display: starts playing the audio from that exact position until the end of the file - **Option**: Timing/Tap To Time: enables the tap marker indicator and commands - **Button**: time_opt_tap_to_time: toggles the Timing/Tap To Time option - **Button**: time_tap_connect (hotkey I): a command that: - moves the tap marker's position to the current playing audio position - sets the next marker to be the tap marker - if the tap marker is already the last marker AND BOTH autocommit AND next-line-on-commit is ON, will move onto the next line - if moved on to the next line, also sets the start marker to the current audio position, so the two lines are connected, and moves to the next tap marker (essentially reinvoking time_tap_connect once) - **Button**: time_tap_no_connect (hotkey O): similar to time_tap_connect, except it will not set the next line's start position even if moved to the next line Expected workflow: 1) User loads song lyrics 2) User splits each line into syllables 3) User turns on tap-to-time, autocommit, and next-line-on-commit 4) User plays audio from beginning, tapping time_tap_connect to each syllable, occasionally tapping time_tap_no_connect when a break between lines is desired 5) If user messes up a line, they can set the tap marker to where they want to restart from, and ctrl-right-click to start the audio a few seconds before it 6) Syllables can be split/merged at will, and adjustments to timing can be done using normal karaoke timing controls
2018-10-21 09:42:33 +02:00
void OnTapMarkerChanged();
public:
/// Constructor
/// @param parent Parent window
/// @param c Project context
AudioKaraoke(wxWindow *parent, agi::Context *c);
/// Destructor
~AudioKaraoke();
/// Is karaoke mode currently enabled?
bool IsEnabled() const { return enabled; }
/// Enable or disable karaoke mode
void SetEnabled(bool enable);
};