{"id":5335,"date":"2023-06-22T14:49:09","date_gmt":"2023-06-22T12:49:09","guid":{"rendered":"https:\/\/cnmlab.fr\/?p=5335"},"modified":"2023-06-22T15:09:15","modified_gmt":"2023-06-22T13:09:15","slug":"the-discoverability-index-streaming-and-content-diversity","status":"publish","type":"post","link":"https:\/\/cnmlab.fr\/en\/short-wave\/the-discoverability-index-streaming-and-content-diversity\/","title":{"rendered":"The Discoverability Index: Streaming and Content Diversity"},"content":{"rendered":"\n

Introduction<\/h2>\n\n\n\n

In a recording industry notable for an overabundance of options, concerns are arising not only about consumers\u2019 difficulty with decision-making but also about the concentration of these options on a small number of titles. These two phenomena can be explained by the new curation methods used by the online streaming platforms. Consequently, in order to maintain the cultural diversity of music available for streaming and to counterbalance certain effects of automated recommendations that have a knowing or unknowing bias, one approach consists in measuring and promoting the propensity for content to be discovered by users\u2014a concept known as \u201cdiscoverability.\u201d<\/p>\n\n\n\n

However, this measurement is contingent upon precise documentation and adequate indexing of the content in question.1<\/a><\/sup>This process, also called \u201cpreconditioning,\u201d is the foundation for the decision-making processes in algorithmic recommendation systems.<\/span> Successful completion of that work is dependent upon several critical factors, including the adoption of industry best practices, standards, and common rules in support of findability and traceability, as well as data governance to facilitate the use and protection of consumers\u2019 and artists\u2019 personal data, using transparent methods that are in compliance with regulations. The intent of this article, which deals with various types of institutional approaches and suggests potential tools and solutions, is to shine a light on the issues of discoverability and the functionality of algorithms. We are calling for a standardization of industry processes and encouraging a regulation of streaming, a broadcast mode that is still in development.<\/p>\n\n\n\n

<\/a>The Challenges of Discoverability in a Fast-Changing Environment<\/h2>\n\n\n\n

Discoverability<\/h3>\n\n\n\n

Our understanding of discoverability is in continual development. One widely accepted definition is derived from the Report of the Joint France\u2013Quebec Mission on Online Discoverability of French-Language Cultural Content (RMFQD):<\/p>\n\n\n\n

\u201cThe discoverability of a piece of content in the digital environment2<\/a><\/sup>We consider the term \u201cdigital environment,\u201d used in the definition given by the joint France\u2013Quebec mission, to include all forms of online public provision of digitized content.<\/span> refers to its availability online and its ability to be found amid a vast array of other content, especially by a person who was not specifically searching for it3<\/a><\/sup>\u201cRapport de la mission franco-qu\u00e9b\u00e9coise sur la d\u00e9couvrabilit\u00e9 en ligne des contenus culturels francophones,\u201d Quebec City: Ministry of Culture and Communications of Quebec and Ministry of Culture of France, 2020, p. 8. https:\/\/cdn-contenu.quebec.ca\/cdn-contenu\/adm\/min\/culture-communications\/publications-adm\/rapport\/Decouvrabilite-Rapport.pdf<\/a>.<\/span>.\u201d<\/p>\n\n\n\n

Energy is now being focused on best practices for stimulating and measuring discoverability.[4]<\/p>\n\n\n\n

Among the initiatives that speak to the heightened interest in this issue was a colloquy held in the French Senate on October 3, 2022, on the diversity of cultural content in languages other than English on digital platforms.4<\/a><\/sup>The symposium in the French Senate is available for viewing online: http:\/\/videos.senat.fr\/video.3007738_633abd7fd0f89.decouvrabilite-des-uvres-dexpression-francophone-sur-plateformes-numeriques-apres-midi<\/a> .<\/span>That event presented an opportunity to call upon the institutions in charge of public policies to take a role in implementing open-access databases that inventory the cultural artifacts being marketed online.<\/p>\n\n\n\n

At least two Canadian initiatives are in line with this proposition: Bill C-11 on online streaming 5<\/a><\/sup>See the Online Streaming Act, Bill C-11 441. House of Commons of Canada, \u201cBill C-11: An Act to amend the Broadcasting Act and to make related and consequential amendments to other Acts,\u201d 2022: https:\/\/www.parl.ca\/DocumentViewer\/en\/44-1\/bill\/C-11\/first-reading<\/a>.<\/span> and the MetaMusic metadata capture project.6<\/a><\/sup>See the official website of the M\u00e9taMusique project: https:\/\/metamusique.ca\/a-propos<\/a><\/span> Let us emphasize the importance of mobilizing metadata in the documentation of content distributed on the platforms. Metadata are used by the online services\u2019 algorithms to contextualize and cross-reference works with the criteria used by the recommendation tools. <\/p>\n\n\n\n

Meanwhile, LATICCE recently launched its \u201cEcho Chamber Research Project: Enriched Metadata and Echo Chambers,\u201d in collaboration with the Mitacs program  and the independent Quebec-based record label InTempo Musique. The project is aimed at more precisely defining the contours of content documentation through the use of metadata and ascertaining the increase in visibility stemming from the use of so-called \u201cenriched\u201d metadata.<\/p>\n\n\n\n

All these initiatives are coinciding and making a mark at a time when speculation is rife regarding the near-future roles of recommendation systems, automated decision-making aids, algorithms, and artificial intelligence programs of all kinds in the choices we make about our event attendance and consumption of cultural products, as well as in the creation of future works.<\/p>\n\n\n\n

However, our understanding of discoverability remains imprecise. Once we assert that there is a challenge to finding music in the context of a \u201chyper-offer,\u201d or saturated marketplace, the issue raises questions of a technical nature, as well as issues surrounding trade secrets, data governance, etc. LATICCE conducted a literature review on the topic 7<\/a><\/sup>Jean-Robert Bisaillon, \u201c\u00catre ou ne pas \u00eatre d\u00e9couvrable\u202f? Une revue de litt\u00e9rature,\u201d Montreal, LATICCE (2022). https:\/\/ceim.uqam.ca\/db\/spip.php?page=article-ceim&id_article=13145<\/a>.<\/span>with a view to determining the main parameters, some of which are restated here. The objective is to refine our understanding of discoverability for the audio recording industry, to take stock of current knowledge, and above all to identify the areas of focus and the research questions that are most likely to guide our efforts in the most rewarding directions. <\/p>\n\n\n\n

<\/a>Discoverability Index<\/h3>\n\n\n\n

LATICCE has produced a prototype discoverability measurement index based on an equation in which P refers to presence, V refers to visibility, and R represents recommendation. This last variable can be further broken down into (c) concordance, (p) pertinence, and (n) novelty. Concordance corresponds precisely with the study subject\u2019s reference shopping cart\u2014in other words, exact correspondence with certain variables in the person\u2019s listening history, especially the list of artists to whose work they have listened. Pertinence refers to a connection with various similar artists, as determined by collaborative filtering. Novelty refers to the presence of proposed items released within the previous 30 days, as corroborated by the lists of new releases published by the Association qu\u00e9b\u00e9coise de l\u2019industrie du disque, du spectacle et de la vid\u00e9o (ADISQ)8<\/a><\/sup>Mich\u00e8le Rioux (ed.), \u201c\u00catre ou ne pas \u00eatre d\u00e9couvrable\u202f? Pr\u00e9sence, visibilit\u00e9 et recommandation des propositions culturelles en ligne. La musique et l\u2019audiovisuel,\u201d public scientific report, UQAM-CEIM-LATICCE (2021). https:\/\/ceim.uqam.ca\/db\/spip.php?page=article-ceim&id_article=13145<\/a>.<\/span>.<\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

These projects are attempting to measure not the effective consumption of digital feeds, as we would when drawing up a commercial chart, but to measure the actions taken in order to display certain content to certain audiences. Aggregated statistics regarding effective consumption (hit lists) or fraudulent practices (fake streams 9<\/a><\/sup>\u201cManipulation des \u00e9coutes en ligne,\u201d Centre national de la musique (2023). https:\/\/cnm.fr\/faux-streams-vrai-phenomene-le-cnm-avec-les-professionnels-pour-lutter-contre-la-fraude<\/a>.fr\/faux-streams-vrai-phenomene-le-cnm-avec-les-professionnels-pour-lutter-contre-la-fraude.<\/span>) tell us little about the algorithmic approaches to content provision or about barriers to discoverability. The discoverability index constitutes a measurement paradigm of a different sort.<\/p>\n\n\n\n

The index is a time-specific measurement that does not indicate the level of discoverability in an absolute sense, but rather indicates the longitudinal evolution (through taking periodic, regular, staggered measurements) of the observed variables, making it possible to estimate the progress and setbacks of the recommendation tools offered by the platforms. It is a technologically neutral tool that gauges the success of the recommendation without suggesting thresholds to be met or indicating consumption trends.<\/p>\n\n\n\n

<\/a>Institutional Approaches and Perspectives<\/h2>\n\n\n\n

<\/a>National Inventory and Presence Among Offerings<\/h3>\n\n\n\n

The presence of works and repertoires within online catalogs is a key variable in evaluating compliance over time with potential regulations promoting diversity. Without presence, diversity and discoverability are obviously compromised.  Presence must be measured in light of national inventories, on a country-by-country basis. In our view, this is embodied in databases\u2014ideally, open and interconnected databases\u2014containing enriched administrative and documentary metadata, listing the cultural artifacts made available online. Thus the formation of a national inventory of works and the associated general-interest (meta)data (GID) is a prerequisite for ensuring verification of presence and facilitating adjustments to that metric.10<\/a><\/sup>See Jo\u00eblle Farchy and Juliette Denis, La culture des donn\u00e9es. Intelligence artificielle et algorithmes dans les industries culturelles<\/em> (Paris: Presses des Mines, 2020), and Claudine Duchesne, Laurent Cytermann, Laurent Vachey, Mathieu Morel, and Tristan Aureau, \u201cRapport relatif aux donn\u00e9es d\u2019int\u00e9r\u00eat g\u00e9n\u00e9ral,\u201d CGE\/IGF (2015). https:\/\/www.economie.gouv.fr\/files\/files\/PDF\/DIG-Rapport-final2015-09.pdf<\/a>.<\/span><\/p>\n\n\n\n

Because it creates a public policy issue, this type of initiative represents a national undertaking. We must consider the role of legal copyright deposit, once it is enhanced to include sensitive patrimonial interest metadata in the context of a digital economy. In this respect, let us highlight the National Library of Finland\u2019s initiative to combine the efforts of all collective rights managers in the country on the topic of attributing ISNI (International Standard Name Identifier) to creators and arts organizations as a bridging identifier.11<\/a><\/sup>National Library of Finland, \u201cISNI Project Launched in Copyright Management Organizations\u201d (2022). www.kansalliskirjasto.fi\/en\/news\/isni-project-launched-copyright-management-organizations<\/a>.<\/span><\/p>\n\n\n\n

Canada and Quebec form one of the three parties (the others being Germany and South Korea) that have signed on to date in support of implementing the Convention on the Protection and Promotion of the Diversity of Cultural Expressions.12<\/a><\/sup>See the roadmaps or the full set of statements addressed to the French Senate on October 3, 2022, as well as V\u00e9ronique Gu\u00e8vremont\u2019s statement at Conference No. 4, \u201cVers un nouvel instrument juridique de l\u2019UNESCO sur la diversit\u00e9 linguistique des contenus culturels en ligne. La formation d\u2019une coalition internationale.\u201d https:\/\/praxis.encommun.io\/n\/Wb5v6Zf1xBuniCjMGY27oINmrhQ\/<\/a>. See also V\u00e9ronique Gu\u00e8vremont et al.<\/em>, \u201cLes mesures de d\u00e9couvrabilit\u00e9 des contenus culturels francophones dans l\u2019environnement num\u00e9rique\u202f: compte rendu des tendances et recommandations,\u201d Chaire UNESCO sur la diversit\u00e9 des expressions culturelles (2019). http:\/\/www.unescodec.chaire.ulaval.ca\/sites\/unescodec.chaire.ulaval.ca\/files\/rapport-decouvrabilite-10_decembre_2019_-_final.pdf<\/a>.<\/span> As it appears that France will join them in the near future, the question of best practices and technical measures to be implemented in order to stimulate and evaluate the discoverability of online content will be an issue of public policy in the years ahead.<\/p>\n\n\n\n

The Canadian Radio-television and Telecommunications Commission (CRTC), an independent public organization with a mandate to regulate and oversee broadcasting and telecommunications in Canada (equivalent to ARCOM in France), is mentioned in the so-called \u201cYale Report\u201d (formally \u201cCanada\u2019s Communications Future: Time to Act\u201d)13<\/a><\/sup>Innovation Canada, \u201cCanada\u2019s Communications Future: Time to Act (Yale Report),\u201d Ottawa, 2020: https:\/\/ised-isde.canada.ca\/site\/broadcasting-telecommunications-legislative-review\/en\/canadas-communications-future-time-act<\/a>.<\/span> and is responsible for monitoring online content offerings in Canada.<\/p>\n\n\n\n

In December 2022, the CRTC completed the process of reviewing its broadcasting policy, and some analyses suggest that this regulatory body has understood the importance of its role with regard to establishing a national inventory. There is an evident willingness to develop an open (meta)database to enable the identification of works.<\/p>\n\n\n\n

The Commission is currently developing a digital monitoring system and an open database to simplify and automatize the process of identifying musical selections. This system relies on probative data [. . .] such as the International Standard Recording Code (ISRC), the International Standard Name Identifier (ISNI) and the International Standard Musical Work Code (ISWC), codes that can be used to confirm the accuracy of the information for any musical selection. [. . .] Once it is made public, this database will greatly facilitate identification of Canadian musical selections and mitigate the risks of non-compliance with regulatory requirements. 14<\/a><\/sup>\u201cRevised Commercial Radio Policy\u201d (Ottawa: Canadian Radio-television and Telecommunications Commission, 2022). https:\/\/crtc.gc.ca\/eng\/archive\/2022\/2022-332.htm<\/a>.<\/span><\/p>\n\n\n\n

Once the construction and use of a database of works is addressed, however, the quality, reliability, and exhaustiveness of that data remain an open question.<\/p>\n\n\n\n

<\/a>Commons, Unique Identifiers, and Interoperability<\/h3>\n\n\n\n

Questions have been raised for years regarding the official prescription, exhaustiveness, and degree of reliability of a database of cultural and patrimonial artifacts from the music sector.<\/p>\n\n\n\n

A number of private initiatives, as well as crowdsourced digital commons initiatives, have addressed this matter. I have been maintaining a list of music metadatabases since 2010; it now has over 100 entries.15<\/a><\/sup>List of 100 music databases and unique identifiers in this sector: bit.ly\/musicalmetadata<\/a><\/span>To date, no database has emerged as a common, reliable, and authoritative source of information. Moreover, few points of connection have been established among the various initiatives.<\/p>\n\n\n\n

However, the Wikidata digital commons offers the option of entering and searching 7,709 types of unique standardized identifiers\u2014machine-readable codes that make it possible to deduplicate and cross-reference works and\/or artists. 16<\/a><\/sup>\u201cList of Properties,\u201d Wikidata, accessed January 2, 2023, https:\/\/www.wikidata.org\/w\/index.php?limit=7000&title=Special:ListProperties\/external-id<\/a>.<\/span> This cross-referencing is helpful in distinguishing among works and attributing them to the correct artists and other rightsholders. Such identifiers exist for the vast majority of works of intellectual property, such as ISBN for books and ISRC for recorded music.17<\/a><\/sup>Unique identifiers are machine-readable codes that meet standards established by specialized organizations such as the International Organization for Standardization (ISO). See, for example, the International Standard Recording Code (ISRC) for audio recordings: https:\/\/www.iso.org\/standard\/9515.html<\/a>.<\/span><\/p>\n\n\n\n

The Wikidata identifier for a work, 18<\/a><\/sup>See, for example, WKID Q925657: https:\/\/www.wikidata.org\/wiki\/Q925657<\/a>.<\/span> a person, or a legal entity thus becomes, by extension, a bridging identifier, facilitating interoperability among artistic contributors and repertoires and the unequivocal matching of works and their creators. In the Finnish example, ISNI functions as a bridging identifier to make such matches possible. A bridging identifier is a unique, standardized identifier that enables cross-referencing in order to remove any ambiguity and allow for the work of discernment and possible deduplication.<\/p>\n\n\n\n

By the same token, the crowdsourced MusicBrainz database is an extremely rich source of musical metadata that must be incorporated into the dynamics of work and of the search for authoritativeness and exhaustiveness. Because it is created and maintained by amateurs (much like Wikipedia and Wikidata), the industry has tended to underestimate its impact. In the effort to construct unequivocal, high-quality data, however, no source ought to be overlooked. At a later stage, it will be necessary to establish suitable protocols for the cross-referencing, verification, and fusion of data.<\/p>\n\n\n\n

<\/a>Open Data<\/h3>\n\n\n\n

Aside from unique identifiers, the open and linked database Wikidata describes a multitude of properties associated with each listed element. Examples include the language or geographic origin of a work or person. It is possible to create a list of all items meeting specific criteria by conducting a search with the SPARQL query language. Property P407, for example, indicates the language associated with a work.<\/p>\n\n\n\n

Let us take as an example a query for a musical composition\u2014a song or piece of music with vocals\u2014created by a French-speaking songwriter or performer born in Quebec or France. This query generates 3,373 results in 52 milliseconds.19<\/a><\/sup>According to the following query made on January 2, 2023: w.wiki\/5Dg8<\/a>.<\/span> The number of titles in open, public song catalogs that are adequately labeled as being of French or Quebec\/Canadian origin and in the French language is well below the expected number.20<\/a><\/sup>LATICCE is currently considering whether to initiate or promote a \u201cdatathon\u201d focused on enrichment of Property P407 in order to study the potential for collective engagement around the issues of contribution to an open database of recorded French-language music. The CRTC affirms that it, too, is working to develop an open database that would make it possible to attribute a language and Canadian origin to a musical selection.<\/span><\/p>\n\n\n\n

As we have just seen in the SPARQL query to Wikidata, an open database must, by definition, be automatically \u201charvestable\u201d so that it can be reused by industry stakeholders or by the public institutions charged with monitoring broadcasters\u2019 compliance with content commitments. Over time, open data offers rightsholders the opportunity to ensure the quality of the data, to confirm or correct the data, and to have remedies available for that purpose.21<\/a><\/sup>By way of example, see the landing page for ISNI database correction requests: isni.oclc.org\/xslt\/DB=1.2\/SET=1\/TTL=1\/WEBCAT?CI_FORMINDEX_COMMAND=update&cmd=update&PPN=466242395&formcode=ISNI-COMMENT&CLT=ISNI<\/a>.<\/span><\/p>\n\n\n\n

<\/a>Indexing at the Source<\/h3>\n\n\n\n

Much like the National Library of Finland\u2019s scheme, the MetaMusic project is based on a joint initiative of all the rights management organizations and unions in the Quebec music industry, in the form of a nonprofit consortium.22<\/a><\/sup>This project was inspired in part by a 2013 paper emphasizing the need for artists and rightsholder to contribute directly to the establishment of a reliable open database: Jean-Robert Bisaillon, \u201cM\u00e9tadonn\u00e9es et politique num\u00e9rique du r\u00e9pertoire musical qu\u00e9b\u00e9cois. Un essai de mobilisation des connaissances dans le nouvel environnement num\u00e9rique\u201d (master\u2019s thesis, Universit\u00e9 du Qu\u00e9bec \u00e0 Montr\u00e9al, 2013).<\/span><\/p>\n\n\n\n

Practice now imposes preconditioning rules on theory, in order to minimize the barriers to discoverability. That is precisely what MetaMusic could make possible: encouraging new best practices with regard to preconditioning in order to ensure that the artifacts distributed through the value chain and via the platforms are adequately documented. Otherwise, algorithmic recommendation systems could overlook this content.<\/p>\n\n\n\n

A good deal has been written about the digital online listening platforms\u2019 recommendation algorithms, in particular by Brian Whitman, co-founder of The Echo Nest, a recommendation project acquired by Spotify in March 2014. 23<\/a><\/sup>Brian Whitman, \u201cComment fonctionne la recommandation musicale?,\u201d Medium saignant (blog), 2019:  http:\/\/mediumsaignant.media\/comment-fonctionne-la-recommandation-musicale\/<\/a>.<\/span> Notably, Whitman addresses the limitations of collaborative filtering, which tends to produce filter bubbles and uninspired recommendations.<\/p>\n\n\n\n

The consulting company Music Tomorrow describes industry practices aimed at a better understanding of recommendation dynamics using the term \u201crecommender system optimization\u201d (RSO), a reference to SEO (search engine optimization), which is aimed at improving organic results in web searches. RSO\u2014particularly technical<\/em> RSO\u2014deals with administrative and documentary metadata, \u201coptimizing the technical distribution pipeline to ensure that the artist catalog and metadata attached are complete and accurate.\u201d24<\/a><\/sup>Dmitry Pastukhov, \u201cTowards Recommender System Optimization: How Can Artists Influence the Spotify Algorithm?,\u201d Music Tomorrow (blog), 2022: https:\/\/www.music-tomorrow.com\/blog\/towards-recommender-system-optimization-how-can-artists-influence-recommendation-algorithms<\/a>.<\/span><\/p>\n\n\n\n

However, we do not yet know precisely which data are key to the promotion of optimal discoverability and, over time, an accurate rendering of accounts. While they may resemble general-interest data (GID), descriptive metadata appear to represent an area in which competition continues.<\/p>\n\n\n\n

<\/a>Tools and Solutions Developed by the LATICCE Team<\/h2>\n\n\n\n

<\/a>Filter Bubbles and Enriched Metadata<\/h3>\n\n\n\n

This is where LATICCE\u2019s current work on enriched metadata comes into play. There exist Electronic Data Interchange (EDI) standards to fuel the music value chain. The Digital Data Exchange (DDEX) consortium was created to develop the electronic protocols needed for data transmission between each segment of the value chain for audio recordings and podcasts. The professional practices of non-Anglophone culture milieus absolutely must be based on this type of international standard.<\/p>\n\n\n\n

The DDEX-MEAD choreography now makes it possible to share Media Enrichment and Description (MEAD) data among producers, digital distributors, and platforms. The MEAD format supports 30 categories of enriched data and an allowed value set (AVS), which can now be exchanged for each of those categories. <\/p>\n\n\n\n

Our project is being developed in conjunction with the Quebec-based label InTempo Musique and several French research partners, including LabEx-ICCA, F\u00c9LIN (a federation of independent labels), and Musicovery. It is aimed at facilitating the analysis of the relationship among the existence of enriched metadata, recommendation quality, and the potential to break through echo chambers or filter bubbles.<\/p>\n\n\n\n

When launched, the project established enriched metadata as follows:<\/p>\n\n\n\n