Refactor enrich CLI into pipeline, fetch, and retry modes
This commit is contained in:
51
README.md
51
README.md
@@ -268,15 +268,14 @@ scrbblr report [OPTIONS]
|
||||
--db-path <PATH> Path to the SQLite database
|
||||
|
||||
scrbblr enrich [OPTIONS]
|
||||
--online Fetch metadata and covers from MusicBrainz / iTunes / CAA
|
||||
--force Re-fetch all albums from MusicBrainz (implies --online)
|
||||
--retry-covers Reset the 7-day cooldown for albums missing covers
|
||||
--retry-mpd-genres Reset the 7-day cooldown for MPD albums missing genres (requires --online)
|
||||
--pipeline <MODE> Enrichment stage: mpd, online, or both [default: both]
|
||||
--fetch <WHAT> Online payload: all, covers, or genres [default: all]
|
||||
--retry <MODE> Retry selector: none, covers, genres, all, mpd-genres [default: none]
|
||||
--cover-source <SRC> Cover source policy: itunes-caa or caa [default: itunes-caa]
|
||||
--force Re-fetch all albums from MusicBrainz (online pipeline only)
|
||||
--artist <ARTIST> Limit enrichment to one artist (case-insensitive)
|
||||
--no-itunes Skip iTunes; fall back directly to Cover Art Archive
|
||||
--no-mpd-covers Skip MPD embedded cover extraction
|
||||
--mpd-host <HOST> MPD host for cover extraction [default: localhost]
|
||||
--mpd-port <PORT> MPD port for cover extraction [default: 6600]
|
||||
--mpd-host <HOST> MPD host for MPD pipeline [default: localhost]
|
||||
--mpd-port <PORT> MPD port for MPD pipeline [default: 6600]
|
||||
--db-path <PATH> Path to the SQLite database
|
||||
|
||||
scrbblr repair-mpd-covers [OPTIONS]
|
||||
@@ -298,28 +297,36 @@ scrbblr pin-album [OPTIONS]
|
||||
|
||||
### Enrichment (covers + genres)
|
||||
|
||||
Enrichment runs in two stages:
|
||||
`enrich` has a clean 3-part model:
|
||||
|
||||
- `--pipeline` decides *where* to run enrichment (`mpd`, `online`, `both`)
|
||||
- `--fetch` decides *what* to fetch online (`covers`, `genres`, `all`)
|
||||
- `--retry` decides *which missing rows* should bypass cooldown before online lookup
|
||||
|
||||
Defaults are: `--pipeline both --fetch all --retry none`.
|
||||
|
||||
The two pipelines are:
|
||||
|
||||
**Stage 1 -- MPD embedded covers (offline, no network)**
|
||||
|
||||
By default, `enrich` connects to MPD and extracts embedded cover art from your music files using MPD's `readpicture` command. This is fast, works entirely offline, and only processes albums scrobbled via MPD that don't yet have a cover:
|
||||
By default (or with `--pipeline mpd`), `enrich` connects to MPD and extracts embedded cover art from your music files using MPD's `readpicture` command. This is fast, works entirely offline, and only processes albums scrobbled via MPD that don't yet have a cover:
|
||||
|
||||
```bash
|
||||
scrbblr enrich
|
||||
scrbblr enrich --pipeline mpd
|
||||
```
|
||||
|
||||
**Stage 2 -- Online lookup (MusicBrainz + iTunes + Cover Art Archive)**
|
||||
|
||||
Pass `--online` to also query MusicBrainz for album metadata (MBID, genres) and download covers for albums that still have none after the MPD pass:
|
||||
Use the online pipeline to query MusicBrainz for metadata (MBID, genres) and fetch covers:
|
||||
|
||||
```bash
|
||||
scrbblr enrich --online
|
||||
scrbblr enrich --pipeline online --fetch all
|
||||
```
|
||||
|
||||
Cover art is sourced from **iTunes first** (fast, no rate limiting), then the **Cover Art Archive** as fallback. Pass `--no-itunes` to skip iTunes and use only CAA:
|
||||
Cover art is sourced from **iTunes first** (fast, no rate limiting), then the **Cover Art Archive** as fallback. To use only CAA:
|
||||
|
||||
```bash
|
||||
scrbblr enrich --online --no-itunes
|
||||
scrbblr enrich --pipeline online --fetch covers --cover-source caa
|
||||
```
|
||||
|
||||
When MusicBrainz matching is tricky, enrichment retries with normalised album variants (strips parenthetical suffixes, progressively shortens trailing words), tries artist aliases for symbol-heavy names, and falls back to recording search before giving up.
|
||||
@@ -331,24 +338,26 @@ Genre extraction order:
|
||||
3. Release-group `genres`
|
||||
4. Release-group `tags`
|
||||
|
||||
Automatic enrichment (triggered by `report --html`) uses a 7-day retry cooldown for incomplete cache entries (missing cover or genre) so it doesn't hammer MusicBrainz on every report run. Use `--retry-covers` to reset that cooldown and retry albums missing covers immediately:
|
||||
Automatic enrichment (triggered by `report --html`) uses a 7-day retry cooldown for incomplete cache entries (missing cover or genre) so it doesn't hammer MusicBrainz on every report run.
|
||||
|
||||
Retry missing covers immediately:
|
||||
|
||||
```bash
|
||||
scrbblr enrich --online --retry-covers
|
||||
scrbblr enrich --pipeline online --retry covers
|
||||
```
|
||||
|
||||
If you specifically want to retry **missing genres for MPD-sourced albums only**:
|
||||
Retry missing genres only for MPD-sourced albums:
|
||||
|
||||
```bash
|
||||
scrbblr enrich --online --retry-mpd-genres
|
||||
scrbblr enrich --pipeline online --fetch genres --retry mpd-genres
|
||||
```
|
||||
|
||||
You can combine this with `--artist` to target one artist.
|
||||
|
||||
Use `--force` to re-fetch all albums from scratch (ignores the cooldown entirely):
|
||||
Use `--force` to re-fetch all albums from scratch (online pipeline only):
|
||||
|
||||
```bash
|
||||
scrbblr enrich --online --force
|
||||
scrbblr enrich --pipeline online --fetch all --force
|
||||
```
|
||||
|
||||
Genre normalisation notes:
|
||||
|
||||
161
src/db.rs
161
src/db.rs
@@ -737,7 +737,8 @@ pub struct UncachedAlbum {
|
||||
/// have no `cover_url`. This makes [`uncached_albums`] pick them up again on
|
||||
/// the next enrichment run, bypassing the 7-day cooldown.
|
||||
///
|
||||
/// Used by `enrich --online --retry-covers` to re-attempt cover downloads
|
||||
/// Used by `enrich --pipeline online --retry covers` to re-attempt cover
|
||||
/// downloads
|
||||
/// without forcing a full re-fetch of all albums.
|
||||
pub fn reset_missing_cover_timestamps(conn: &Connection) -> Result<usize> {
|
||||
let count = conn.execute(
|
||||
@@ -762,6 +763,60 @@ pub fn reset_missing_cover_timestamps_for_artist(conn: &Connection, artist: &str
|
||||
Ok(count)
|
||||
}
|
||||
|
||||
/// Reset the `fetched_at` timestamp to NULL for all rows that have no genre.
|
||||
///
|
||||
/// Used when the user explicitly wants to retry genre lookups immediately,
|
||||
/// bypassing the default 7-day cooldown.
|
||||
pub fn reset_missing_genre_timestamps(conn: &Connection) -> Result<usize> {
|
||||
let count = conn.execute(
|
||||
"UPDATE album_cache
|
||||
SET fetched_at = NULL
|
||||
WHERE genre IS NULL",
|
||||
[],
|
||||
)?;
|
||||
Ok(count)
|
||||
}
|
||||
|
||||
/// Reset `fetched_at` to NULL for genre-missing albums for one artist only.
|
||||
///
|
||||
/// Matching is case-insensitive (`LOWER(artist) = LOWER(?1)`).
|
||||
pub fn reset_missing_genre_timestamps_for_artist(conn: &Connection, artist: &str) -> Result<usize> {
|
||||
let count = conn.execute(
|
||||
"UPDATE album_cache
|
||||
SET fetched_at = NULL
|
||||
WHERE genre IS NULL
|
||||
AND LOWER(artist) = LOWER(?1)",
|
||||
params![artist],
|
||||
)?;
|
||||
Ok(count)
|
||||
}
|
||||
|
||||
/// Reset `fetched_at` to NULL for rows missing either cover or genre.
|
||||
pub fn reset_incomplete_timestamps(conn: &Connection) -> Result<usize> {
|
||||
let count = conn.execute(
|
||||
"UPDATE album_cache
|
||||
SET fetched_at = NULL
|
||||
WHERE cover_url IS NULL OR genre IS NULL",
|
||||
[],
|
||||
)?;
|
||||
Ok(count)
|
||||
}
|
||||
|
||||
/// Reset `fetched_at` to NULL for incomplete rows for one artist only.
|
||||
///
|
||||
/// A row is considered incomplete when either `cover_url` or `genre` is NULL.
|
||||
/// Matching is case-insensitive (`LOWER(artist) = LOWER(?1)`).
|
||||
pub fn reset_incomplete_timestamps_for_artist(conn: &Connection, artist: &str) -> Result<usize> {
|
||||
let count = conn.execute(
|
||||
"UPDATE album_cache
|
||||
SET fetched_at = NULL
|
||||
WHERE (cover_url IS NULL OR genre IS NULL)
|
||||
AND LOWER(artist) = LOWER(?1)",
|
||||
params![artist],
|
||||
)?;
|
||||
Ok(count)
|
||||
}
|
||||
|
||||
/// Reset `fetched_at` to NULL for MPD-sourced albums that have no genre yet.
|
||||
///
|
||||
/// This allows online enrichment to re-try genre lookups immediately for
|
||||
@@ -2152,6 +2207,110 @@ mod tests {
|
||||
assert_eq!(deftones_fetched.as_deref(), Some(expected.as_str()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_reset_missing_genre_timestamps_for_artist_only() {
|
||||
let conn = open_memory_db().unwrap();
|
||||
|
||||
upsert_album_cache(
|
||||
&conn,
|
||||
&AlbumCacheEntry {
|
||||
artist: "Pink Floyd".into(),
|
||||
album: "The Wall".into(),
|
||||
musicbrainz_id: Some("mbid-1".into()),
|
||||
cover_url: Some("covers/x.jpg".into()),
|
||||
genre: None,
|
||||
fetched_at: today_at("12:00:00"),
|
||||
},
|
||||
)
|
||||
.unwrap();
|
||||
upsert_album_cache(
|
||||
&conn,
|
||||
&AlbumCacheEntry {
|
||||
artist: "Deftones".into(),
|
||||
album: "White Pony".into(),
|
||||
musicbrainz_id: Some("mbid-2".into()),
|
||||
cover_url: Some("covers/y.jpg".into()),
|
||||
genre: None,
|
||||
fetched_at: today_at("12:01:00"),
|
||||
},
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let updated = reset_missing_genre_timestamps_for_artist(&conn, "pink floyd").unwrap();
|
||||
assert_eq!(updated, 1);
|
||||
|
||||
let pink_fetched: Option<String> = conn
|
||||
.query_row(
|
||||
"SELECT fetched_at FROM album_cache WHERE artist = 'Pink Floyd' AND album = 'The Wall'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
.unwrap();
|
||||
assert!(pink_fetched.is_none());
|
||||
|
||||
let deftones_fetched: Option<String> = conn
|
||||
.query_row(
|
||||
"SELECT fetched_at FROM album_cache WHERE artist = 'Deftones' AND album = 'White Pony'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
.unwrap();
|
||||
let expected = today_at("12:01:00");
|
||||
assert_eq!(deftones_fetched.as_deref(), Some(expected.as_str()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_reset_incomplete_timestamps_for_artist_only() {
|
||||
let conn = open_memory_db().unwrap();
|
||||
|
||||
upsert_album_cache(
|
||||
&conn,
|
||||
&AlbumCacheEntry {
|
||||
artist: "Pink Floyd".into(),
|
||||
album: "The Wall".into(),
|
||||
musicbrainz_id: Some("mbid-1".into()),
|
||||
cover_url: None,
|
||||
genre: Some("progressive rock".into()),
|
||||
fetched_at: today_at("12:00:00"),
|
||||
},
|
||||
)
|
||||
.unwrap();
|
||||
upsert_album_cache(
|
||||
&conn,
|
||||
&AlbumCacheEntry {
|
||||
artist: "Deftones".into(),
|
||||
album: "White Pony".into(),
|
||||
musicbrainz_id: Some("mbid-2".into()),
|
||||
cover_url: Some("covers/y.jpg".into()),
|
||||
genre: None,
|
||||
fetched_at: today_at("12:01:00"),
|
||||
},
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let updated = reset_incomplete_timestamps_for_artist(&conn, "pink floyd").unwrap();
|
||||
assert_eq!(updated, 1);
|
||||
|
||||
let pink_fetched: Option<String> = conn
|
||||
.query_row(
|
||||
"SELECT fetched_at FROM album_cache WHERE artist = 'Pink Floyd' AND album = 'The Wall'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
.unwrap();
|
||||
assert!(pink_fetched.is_none());
|
||||
|
||||
let deftones_fetched: Option<String> = conn
|
||||
.query_row(
|
||||
"SELECT fetched_at FROM album_cache WHERE artist = 'Deftones' AND album = 'White Pony'",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
.unwrap();
|
||||
let expected = today_at("12:01:00");
|
||||
assert_eq!(deftones_fetched.as_deref(), Some(expected.as_str()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_reset_missing_genre_timestamps_for_mpd_only() {
|
||||
let conn = open_memory_db().unwrap();
|
||||
|
||||
127
src/enrich.rs
127
src/enrich.rs
@@ -13,8 +13,8 @@
|
||||
//! 2. For each, searches MusicBrainz for matching releases (to get MBID and
|
||||
//! genre/tag data — always done regardless of cover source).
|
||||
//! 3. Tries iTunes Search API first for cover art (fast, no rate limit,
|
||||
//! good commercial coverage). Pass `--no-itunes` to skip.
|
||||
//! 4. If iTunes has no cover (or `--no-itunes`), falls back to the Cover Art
|
||||
//! good commercial coverage). Can be switched to CAA-only mode.
|
||||
//! 4. If iTunes has no cover (or iTunes is disabled), falls back to the Cover Art
|
||||
//! Archive using the MBID from step 2:
|
||||
//! a. The specific release endpoint
|
||||
//! b. The release-group endpoint (covers any edition of the album)
|
||||
@@ -97,6 +97,52 @@ const MAX_COVER_DIM: u32 = 500;
|
||||
/// JPEG quality used when re-encoding resized cover art (0–100).
|
||||
const COVER_JPEG_QUALITY: u8 = 85;
|
||||
|
||||
/// What the online enrichment stage should fetch.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum OnlineFetchMode {
|
||||
/// Fetch genres and covers.
|
||||
All,
|
||||
/// Fetch covers only.
|
||||
Covers,
|
||||
/// Fetch genres only.
|
||||
Genres,
|
||||
}
|
||||
|
||||
impl OnlineFetchMode {
|
||||
fn wants_covers(self) -> bool {
|
||||
matches!(self, Self::All | Self::Covers)
|
||||
}
|
||||
|
||||
fn wants_genres(self) -> bool {
|
||||
matches!(self, Self::All | Self::Genres)
|
||||
}
|
||||
}
|
||||
|
||||
/// Cover source policy for the online stage.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum CoverSourceMode {
|
||||
/// Try iTunes first, then fall back to Cover Art Archive.
|
||||
ItunesThenCaa,
|
||||
/// Skip iTunes and use only Cover Art Archive.
|
||||
CaaOnly,
|
||||
}
|
||||
|
||||
/// Options controlling online enrichment behaviour.
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub struct OnlineEnrichOptions {
|
||||
pub fetch_mode: OnlineFetchMode,
|
||||
pub cover_source: CoverSourceMode,
|
||||
}
|
||||
|
||||
impl Default for OnlineEnrichOptions {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
fetch_mode: OnlineFetchMode::All,
|
||||
cover_source: CoverSourceMode::ItunesThenCaa,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// MusicBrainz API response types (only the fields we need)
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -959,9 +1005,8 @@ pub fn run_enrich_targeted(
|
||||
conn: &Connection,
|
||||
needed: &std::collections::HashSet<(String, String)>,
|
||||
quiet: bool,
|
||||
no_itunes: bool,
|
||||
) {
|
||||
run_enrich_targeted_with_options(conn, needed, quiet, no_itunes, false);
|
||||
run_enrich_targeted_with_options(conn, needed, quiet, false, OnlineEnrichOptions::default());
|
||||
}
|
||||
|
||||
/// Like [`run_enrich_targeted`], but supports force refresh for the provided
|
||||
@@ -970,8 +1015,8 @@ pub fn run_enrich_targeted_with_options(
|
||||
conn: &Connection,
|
||||
needed: &std::collections::HashSet<(String, String)>,
|
||||
quiet: bool,
|
||||
no_itunes: bool,
|
||||
force: bool,
|
||||
options: OnlineEnrichOptions,
|
||||
) {
|
||||
if needed.is_empty() {
|
||||
if !quiet {
|
||||
@@ -993,7 +1038,7 @@ pub fn run_enrich_targeted_with_options(
|
||||
"Force mode enabled: refreshing {} selected album(s).",
|
||||
albums.len()
|
||||
);
|
||||
run_enrich_albums(conn, albums, quiet, no_itunes, true);
|
||||
run_enrich_albums(conn, albums, quiet, true, options);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1009,7 +1054,7 @@ pub fn run_enrich_targeted_with_options(
|
||||
.into_iter()
|
||||
.filter(|a| needed.contains(&(a.artist.clone(), a.album.clone())))
|
||||
.collect();
|
||||
run_enrich_albums(conn, albums, quiet, no_itunes, false);
|
||||
run_enrich_albums(conn, albums, quiet, false, options);
|
||||
}
|
||||
|
||||
/// Run the enrichment process: find all uncached albums and fetch their
|
||||
@@ -1023,8 +1068,13 @@ pub fn run_enrich_targeted_with_options(
|
||||
/// - `force` — if true, re-fetch all albums even if already cached
|
||||
/// - `quiet` — if true, suppress the "nothing to do" hint (used when called
|
||||
/// from `report --html` where the user didn't explicitly ask for enrichment)
|
||||
/// - `no_itunes` — if true, skip the iTunes cover lookup and go straight to CAA
|
||||
pub fn run_enrich(conn: &Connection, force: bool, quiet: bool, no_itunes: bool) {
|
||||
/// Run online enrichment with explicit fetch/source options.
|
||||
pub fn run_enrich_with_options(
|
||||
conn: &Connection,
|
||||
force: bool,
|
||||
quiet: bool,
|
||||
options: OnlineEnrichOptions,
|
||||
) {
|
||||
if force {
|
||||
conn.execute("DELETE FROM album_cache", [])
|
||||
.expect("Failed to clear album_cache");
|
||||
@@ -1032,7 +1082,7 @@ pub fn run_enrich(conn: &Connection, force: bool, quiet: bool, no_itunes: bool)
|
||||
eprintln!("Force mode enabled: refreshing existing cover files when re-fetching.");
|
||||
}
|
||||
let albums = db::uncached_albums(conn).expect("Failed to query uncached albums");
|
||||
run_enrich_albums(conn, albums, quiet, no_itunes, force);
|
||||
run_enrich_albums(conn, albums, quiet, force, options);
|
||||
}
|
||||
|
||||
/// Inner enrichment loop shared by `run_enrich` and `run_enrich_targeted`.
|
||||
@@ -1040,8 +1090,8 @@ fn run_enrich_albums(
|
||||
conn: &Connection,
|
||||
albums: Vec<db::UncachedAlbum>,
|
||||
quiet: bool,
|
||||
no_itunes: bool,
|
||||
force_redownload: bool,
|
||||
options: OnlineEnrichOptions,
|
||||
) {
|
||||
if albums.is_empty() {
|
||||
if !quiet {
|
||||
@@ -1053,11 +1103,15 @@ fn run_enrich_albums(
|
||||
|
||||
let client = build_client();
|
||||
let covers = covers_dir();
|
||||
let fetch_covers = options.fetch_mode.wants_covers();
|
||||
let fetch_genres = options.fetch_mode.wants_genres();
|
||||
let use_itunes = fetch_covers && matches!(options.cover_source, CoverSourceMode::ItunesThenCaa);
|
||||
|
||||
eprintln!("Found {} album(s) to enrich.", albums.len());
|
||||
|
||||
let mut success_count = 0;
|
||||
let mut cover_count = 0;
|
||||
let mut genre_count = 0;
|
||||
|
||||
for (i, album) in albums.iter().enumerate() {
|
||||
eprintln!(
|
||||
@@ -1068,6 +1122,12 @@ fn run_enrich_albums(
|
||||
album.album
|
||||
);
|
||||
|
||||
let existing = db::album_cache_meta(conn, &album.artist, &album.album)
|
||||
.ok()
|
||||
.flatten();
|
||||
let existing_genre = existing.as_ref().and_then(|m| m.genre.clone());
|
||||
let existing_mbid = existing.as_ref().and_then(|m| m.mbid.clone());
|
||||
|
||||
// Step 1: Search MusicBrainz for matching releases.
|
||||
// Always done — we need the MBID for genre data and as the CAA
|
||||
// lookup key, even if we end up getting the cover from iTunes.
|
||||
@@ -1079,7 +1139,9 @@ fn run_enrich_albums(
|
||||
|
||||
// Without an MBID we can't query CAA, so iTunes is the only
|
||||
// cover option. Use a hash-based filename since there's no MBID.
|
||||
let cover = if no_itunes {
|
||||
let cover = if !fetch_covers {
|
||||
None
|
||||
} else if !use_itunes {
|
||||
eprintln!(" iTunes disabled, no cover available.");
|
||||
None
|
||||
} else {
|
||||
@@ -1094,7 +1156,7 @@ fn run_enrich_albums(
|
||||
if cover.is_some() {
|
||||
eprintln!(" Cover downloaded (from iTunes).");
|
||||
cover_count += 1;
|
||||
} else if !no_itunes {
|
||||
} else if fetch_covers && use_itunes {
|
||||
eprintln!(" No cover art available from any source.");
|
||||
}
|
||||
|
||||
@@ -1102,9 +1164,9 @@ fn run_enrich_albums(
|
||||
let entry = AlbumCacheEntry {
|
||||
artist: album.artist.clone(),
|
||||
album: album.album.clone(),
|
||||
musicbrainz_id: None,
|
||||
musicbrainz_id: existing_mbid,
|
||||
cover_url: cover,
|
||||
genre: None,
|
||||
genre: existing_genre,
|
||||
fetched_at: now_str(),
|
||||
};
|
||||
match db::upsert_album_cache(conn, &entry) {
|
||||
@@ -1119,16 +1181,26 @@ fn run_enrich_albums(
|
||||
eprintln!(" Found MBID: {}", primary_mbid);
|
||||
|
||||
// Step 2: Fetch genres/tags and release-group ID from the primary release.
|
||||
let (fetched_genre, release_group_id) = if fetch_genres || fetch_covers {
|
||||
thread::sleep(RATE_LIMIT_DELAY);
|
||||
let (genre, release_group_id) = fetch_release_details(&client, primary_mbid);
|
||||
if let Some(ref g) = genre {
|
||||
fetch_release_details(&client, primary_mbid)
|
||||
} else {
|
||||
(None, None)
|
||||
};
|
||||
|
||||
if let Some(ref g) = fetched_genre {
|
||||
if fetch_genres {
|
||||
genre_count += 1;
|
||||
}
|
||||
eprintln!(" Genres: {}", g);
|
||||
}
|
||||
|
||||
// Step 3: Try iTunes first — it's fast, has no rate limit, and has
|
||||
// good commercial coverage. Reuse the MBID-based filename so the file
|
||||
// is still associated with the correct release regardless of source.
|
||||
let mut cover = if no_itunes {
|
||||
let mut cover = if !fetch_covers {
|
||||
None
|
||||
} else if !use_itunes {
|
||||
None
|
||||
} else {
|
||||
let itunes_cover = fetch_itunes_cover_url(&client, &album.artist, &album.album)
|
||||
@@ -1145,8 +1217,8 @@ fn run_enrich_albums(
|
||||
// Step 4: If iTunes had no cover, fall back to the Cover Art Archive.
|
||||
// CAA is better for obscure or non-commercial releases, and has the
|
||||
// highest possible resolution for releases it does have.
|
||||
if cover.is_none() {
|
||||
if !no_itunes {
|
||||
if fetch_covers && cover.is_none() {
|
||||
if use_itunes {
|
||||
eprintln!(" No iTunes cover. Trying Cover Art Archive...");
|
||||
}
|
||||
|
||||
@@ -1181,17 +1253,23 @@ fn run_enrich_albums(
|
||||
|
||||
if cover.is_some() {
|
||||
cover_count += 1;
|
||||
} else {
|
||||
} else if fetch_covers {
|
||||
eprintln!(" No cover art available from any source.");
|
||||
}
|
||||
|
||||
let genre_to_store = if fetch_genres {
|
||||
fetched_genre.or(existing_genre)
|
||||
} else {
|
||||
existing_genre
|
||||
};
|
||||
|
||||
// Step 5: Store the result in album_cache.
|
||||
let entry = AlbumCacheEntry {
|
||||
artist: album.artist.clone(),
|
||||
album: album.album.clone(),
|
||||
musicbrainz_id: Some(primary_mbid.clone()),
|
||||
cover_url: cover,
|
||||
genre,
|
||||
genre: genre_to_store,
|
||||
fetched_at: now_str(),
|
||||
};
|
||||
|
||||
@@ -1205,7 +1283,12 @@ fn run_enrich_albums(
|
||||
eprintln!("Enrichment complete:");
|
||||
eprintln!(" Albums processed: {}", albums.len());
|
||||
eprintln!(" Successfully cached: {}", success_count);
|
||||
if fetch_covers {
|
||||
eprintln!(" Covers downloaded: {}", cover_count);
|
||||
}
|
||||
if fetch_genres {
|
||||
eprintln!(" Albums with genres fetched: {}", genre_count);
|
||||
}
|
||||
eprintln!(" Covers directory: {}", covers.display());
|
||||
}
|
||||
|
||||
|
||||
231
src/main.rs
231
src/main.rs
@@ -55,7 +55,7 @@ mod mpd;
|
||||
mod report;
|
||||
mod watcher;
|
||||
|
||||
use clap::{Parser, Subcommand};
|
||||
use clap::{Parser, Subcommand, ValueEnum};
|
||||
use std::io::BufRead;
|
||||
use std::process::{Command, Stdio};
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
@@ -77,6 +77,39 @@ struct Cli {
|
||||
command: Commands,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, ValueEnum)]
|
||||
#[value(rename_all = "kebab-case")]
|
||||
enum EnrichPipeline {
|
||||
Mpd,
|
||||
Online,
|
||||
Both,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, ValueEnum)]
|
||||
#[value(rename_all = "kebab-case")]
|
||||
enum EnrichFetch {
|
||||
All,
|
||||
Covers,
|
||||
Genres,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, ValueEnum)]
|
||||
#[value(rename_all = "kebab-case")]
|
||||
enum EnrichRetry {
|
||||
None,
|
||||
Covers,
|
||||
Genres,
|
||||
All,
|
||||
MpdGenres,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, ValueEnum)]
|
||||
#[value(rename_all = "kebab-case")]
|
||||
enum CoverSource {
|
||||
ItunesCaa,
|
||||
Caa,
|
||||
}
|
||||
|
||||
#[derive(Subcommand)]
|
||||
enum Commands {
|
||||
/// Watch playerctl metadata and MPD, scrobbling tracks to the local database.
|
||||
@@ -164,40 +197,40 @@ enum Commands {
|
||||
},
|
||||
/// Fetch album art and genre info for all scrobbled albums.
|
||||
///
|
||||
/// By default, connects to MPD and extracts embedded cover art from music
|
||||
/// files via `readpicture` — fully offline, no network access required.
|
||||
/// Pass `--no-mpd-covers` to skip this step.
|
||||
/// By default, runs both stages:
|
||||
///
|
||||
/// Pass `--online` to also query MusicBrainz for album metadata (MBID,
|
||||
/// genre) and download cover art. Cover art is sourced from iTunes first
|
||||
/// (fast, no rate limit), then the Cover Art Archive as fallback. Pass
|
||||
/// `--no-itunes` to skip iTunes and use only the Cover Art Archive.
|
||||
/// 1. MPD embedded-cover extraction (`readpicture`, offline)
|
||||
/// 2. Online metadata/cover enrichment (MusicBrainz + iTunes/CAA)
|
||||
///
|
||||
/// Use `--pipeline`, `--fetch`, and `--retry` to narrow the behaviour.
|
||||
Enrich {
|
||||
/// Query MusicBrainz for album metadata (MBID, genre) and download
|
||||
/// cover art from the Cover Art Archive for albums that still have
|
||||
/// no cover after local extraction.
|
||||
#[arg(long)]
|
||||
online: bool,
|
||||
/// Which enrichment stage(s) to run.
|
||||
#[arg(long, value_enum, default_value = "both")]
|
||||
pipeline: EnrichPipeline,
|
||||
|
||||
/// Which metadata to fetch during the online stage.
|
||||
#[arg(long, value_enum, default_value = "all")]
|
||||
fetch: EnrichFetch,
|
||||
|
||||
/// Reset cooldown before online enrichment.
|
||||
///
|
||||
/// - `none`: keep normal 7-day cooldown
|
||||
/// - `covers`: retry cover-missing rows
|
||||
/// - `genres`: retry genre-missing rows
|
||||
/// - `all`: retry rows missing either cover or genre
|
||||
/// - `mpd-genres`: retry genre-missing rows only for MPD-sourced albums
|
||||
#[arg(long, value_enum, default_value = "none")]
|
||||
retry: EnrichRetry,
|
||||
|
||||
/// Cover source policy for the online stage.
|
||||
#[arg(long, value_enum, default_value = "itunes-caa")]
|
||||
cover_source: CoverSource,
|
||||
|
||||
/// Re-fetch metadata for all albums from MusicBrainz, even those
|
||||
/// already cached. Only meaningful together with `--online`.
|
||||
/// already cached. Only meaningful when `--pipeline` includes `online`.
|
||||
#[arg(long)]
|
||||
force: bool,
|
||||
|
||||
/// Re-attempt cover downloads for albums that have metadata cached
|
||||
/// but no cover art yet. Resets the 7-day cooldown for those albums
|
||||
/// only, so the next `--online` run retries them. Does not affect
|
||||
/// albums that already have a cover.
|
||||
#[arg(long)]
|
||||
retry_covers: bool,
|
||||
|
||||
/// Re-attempt genre lookups for MPD-sourced albums that still have no
|
||||
/// genre. Resets the 7-day cooldown for those albums only, so the next
|
||||
/// `--online` run retries them. Does not affect albums that already
|
||||
/// have a genre.
|
||||
#[arg(long)]
|
||||
retry_mpd_genres: bool,
|
||||
|
||||
/// Limit enrichment to albums by this artist (case-insensitive match).
|
||||
///
|
||||
/// Useful when fixing one artist's covers/metadata without touching
|
||||
@@ -206,25 +239,12 @@ enum Commands {
|
||||
#[arg(long)]
|
||||
artist: Option<String>,
|
||||
|
||||
/// Skip the iTunes Search API when fetching cover art. Falls back
|
||||
/// directly to the Cover Art Archive (CAA). Useful if you prefer
|
||||
/// open-data sources or iTunes results are poor for your library.
|
||||
#[arg(long)]
|
||||
no_itunes: bool,
|
||||
|
||||
/// Disable the MPD embedded cover extraction step. By default,
|
||||
/// `enrich` connects to MPD and extracts embedded cover art from
|
||||
/// music files via `readpicture` before doing anything else.
|
||||
#[arg(long)]
|
||||
no_mpd_covers: bool,
|
||||
|
||||
/// MPD server hostname, IP address, or Unix socket path.
|
||||
/// Used for embedded cover extraction (unless `--no-mpd-covers`).
|
||||
/// Used for MPD extraction when `--pipeline` includes `mpd`.
|
||||
#[arg(long, default_value = "localhost")]
|
||||
mpd_host: String,
|
||||
|
||||
/// MPD server TCP port. Used for embedded cover extraction when
|
||||
/// connecting via TCP (ignored for Unix socket paths).
|
||||
/// MPD server TCP port. Ignored for Unix socket paths.
|
||||
#[arg(long, default_value = "6600")]
|
||||
mpd_port: u16,
|
||||
|
||||
@@ -595,7 +615,7 @@ fn run_report(
|
||||
// iTunes is tried first (fast, no rate limit), then CAA as fallback.
|
||||
// Respects the 7-day cooldown so we don't hammer MusicBrainz on every
|
||||
// report run.
|
||||
enrich::run_enrich_targeted(&conn, &needed, true, false);
|
||||
enrich::run_enrich_targeted(&conn, &needed, true);
|
||||
}
|
||||
|
||||
if html {
|
||||
@@ -755,29 +775,38 @@ fn main() {
|
||||
);
|
||||
}
|
||||
Commands::Enrich {
|
||||
online,
|
||||
pipeline,
|
||||
fetch,
|
||||
retry,
|
||||
cover_source,
|
||||
force,
|
||||
retry_covers,
|
||||
retry_mpd_genres,
|
||||
artist,
|
||||
no_itunes,
|
||||
no_mpd_covers,
|
||||
mpd_host,
|
||||
mpd_port,
|
||||
db_path,
|
||||
} => {
|
||||
if retry_mpd_genres && !online {
|
||||
eprintln!("--retry-mpd-genres requires --online.");
|
||||
let run_mpd = matches!(pipeline, EnrichPipeline::Mpd | EnrichPipeline::Both);
|
||||
let run_online = matches!(pipeline, EnrichPipeline::Online | EnrichPipeline::Both);
|
||||
let wants_covers = matches!(fetch, EnrichFetch::All | EnrichFetch::Covers);
|
||||
|
||||
if force && !run_online {
|
||||
eprintln!("--force requires --pipeline online or --pipeline both.");
|
||||
std::process::exit(1);
|
||||
}
|
||||
|
||||
if no_mpd_covers && !online {
|
||||
eprintln!("Nothing to do. Pass --online and/or omit --no-mpd-covers.");
|
||||
if !matches!(retry, EnrichRetry::None) && !run_online {
|
||||
eprintln!("--retry requires --pipeline online or --pipeline both.");
|
||||
std::process::exit(1);
|
||||
}
|
||||
|
||||
if run_mpd && !wants_covers && !run_online {
|
||||
eprintln!(
|
||||
" (default) Extract embedded covers from music files via MPD (offline)"
|
||||
);
|
||||
eprintln!(
|
||||
" --online Fetch metadata and covers from MusicBrainz / iTunes / CAA"
|
||||
"Nothing to do: --pipeline mpd only handles covers, but --fetch {} excludes covers.",
|
||||
match fetch {
|
||||
EnrichFetch::Genres => "genres",
|
||||
EnrichFetch::All => "all",
|
||||
EnrichFetch::Covers => "covers",
|
||||
}
|
||||
);
|
||||
std::process::exit(0);
|
||||
}
|
||||
@@ -816,10 +845,10 @@ fn main() {
|
||||
None
|
||||
};
|
||||
|
||||
// Run MPD cover extraction first (fast, local, no rate limits).
|
||||
// Pre-populating cover_url means the online enrichment that follows
|
||||
// will skip fetching covers for albums that already have one.
|
||||
if !no_mpd_covers {
|
||||
// Run MPD cover extraction first (fast, local, no rate limits)
|
||||
// when the selected pipeline includes MPD and the fetch mode asks
|
||||
// for covers.
|
||||
if run_mpd && wants_covers {
|
||||
let mpd_cfg = mpd::MpdConfig {
|
||||
host: mpd_host,
|
||||
port: mpd_port,
|
||||
@@ -831,45 +860,79 @@ fn main() {
|
||||
}
|
||||
}
|
||||
|
||||
// Online enrichment: MusicBrainz lookup for MBID + genre, Cover Art
|
||||
// Archive for any albums still missing a cover after the MPD pass.
|
||||
// Reset the cooldown for cover-missing albums before the online
|
||||
// pass so they are picked up by uncached_albums.
|
||||
if retry_covers {
|
||||
let reset = if let Some(ref artist_name) = artist {
|
||||
if run_online {
|
||||
// Optional retry selector: reset cooldown timestamps so the
|
||||
// online stage revisits selected missing fields immediately.
|
||||
if !matches!(retry, EnrichRetry::None) {
|
||||
let reset = match retry {
|
||||
EnrichRetry::None => Ok(0),
|
||||
EnrichRetry::Covers => {
|
||||
if let Some(ref artist_name) = artist {
|
||||
db::reset_missing_cover_timestamps_for_artist(&conn, artist_name)
|
||||
} else {
|
||||
db::reset_missing_cover_timestamps(&conn)
|
||||
};
|
||||
|
||||
match reset {
|
||||
Ok(n) => eprintln!("Reset cooldown for {} album(s) missing covers.", n),
|
||||
Err(e) => eprintln!("[warn] Failed to reset cover timestamps: {}", e),
|
||||
}
|
||||
}
|
||||
|
||||
if retry_mpd_genres {
|
||||
let reset = if let Some(ref artist_name) = artist {
|
||||
db::reset_missing_genre_timestamps_for_mpd_artist(&conn, artist_name)
|
||||
EnrichRetry::Genres => {
|
||||
if let Some(ref artist_name) = artist {
|
||||
db::reset_missing_genre_timestamps_for_artist(&conn, artist_name)
|
||||
} else {
|
||||
db::reset_missing_genre_timestamps(&conn)
|
||||
}
|
||||
}
|
||||
EnrichRetry::All => {
|
||||
if let Some(ref artist_name) = artist {
|
||||
db::reset_incomplete_timestamps_for_artist(&conn, artist_name)
|
||||
} else {
|
||||
db::reset_incomplete_timestamps(&conn)
|
||||
}
|
||||
}
|
||||
EnrichRetry::MpdGenres => {
|
||||
if let Some(ref artist_name) = artist {
|
||||
db::reset_missing_genre_timestamps_for_mpd_artist(
|
||||
&conn,
|
||||
artist_name,
|
||||
)
|
||||
} else {
|
||||
db::reset_missing_genre_timestamps_for_mpd(&conn)
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
match reset {
|
||||
Ok(n) => {
|
||||
eprintln!("Reset cooldown for {} MPD album(s) missing genres.", n)
|
||||
Ok(n) => eprintln!(
|
||||
"Reset cooldown for {} album(s) via --retry {}.",
|
||||
n,
|
||||
match retry {
|
||||
EnrichRetry::None => "none",
|
||||
EnrichRetry::Covers => "covers",
|
||||
EnrichRetry::Genres => "genres",
|
||||
EnrichRetry::All => "all",
|
||||
EnrichRetry::MpdGenres => "mpd-genres",
|
||||
}
|
||||
Err(e) => eprintln!("[warn] Failed to reset MPD genre timestamps: {}", e),
|
||||
),
|
||||
Err(e) => eprintln!("[warn] Failed to reset retry timestamps: {}", e),
|
||||
}
|
||||
}
|
||||
|
||||
if online {
|
||||
let cover_source_mode = match cover_source {
|
||||
CoverSource::ItunesCaa => enrich::CoverSourceMode::ItunesThenCaa,
|
||||
CoverSource::Caa => enrich::CoverSourceMode::CaaOnly,
|
||||
};
|
||||
let fetch_mode = match fetch {
|
||||
EnrichFetch::All => enrich::OnlineFetchMode::All,
|
||||
EnrichFetch::Covers => enrich::OnlineFetchMode::Covers,
|
||||
EnrichFetch::Genres => enrich::OnlineFetchMode::Genres,
|
||||
};
|
||||
let options = enrich::OnlineEnrichOptions {
|
||||
fetch_mode,
|
||||
cover_source: cover_source_mode,
|
||||
};
|
||||
|
||||
if let Some(ref needed) = artist_needed {
|
||||
enrich::run_enrich_targeted_with_options(
|
||||
&conn, needed, false, no_itunes, force,
|
||||
);
|
||||
enrich::run_enrich_targeted_with_options(&conn, needed, false, force, options);
|
||||
} else {
|
||||
enrich::run_enrich(&conn, force, false, no_itunes);
|
||||
enrich::run_enrich_with_options(&conn, force, false, options);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user