2024-03-25 White Paper Blog

MetaRex needs you - sign up and make metadata flow!

Bruce Devlin @in

MetaRex needs you

The award winning , philanthropic, non-profit MetaRex project makes metadata flow. The more metadata you have, the more you will benefit. But we need more backers to get the project over the finish line. Ideally we’d like 2 more silver backers or 15 more regular backers ; but first, let’s lay out our objectives:

  1. ๐Ÿ†“ MetaRex is a free software framework
  2. ๐Ÿ˜Ž MetaRex lets you define your own metadata.
  3. ๐Ÿ“ฆ MetaRex defines a container for your metadata.
  4. ๐ŸŒ MetaRex maps containers to many types of network so that you get metadata transport for free!
  5. ๐Ÿ†” MetaRex gives you a unique Id for your new type of metadata.
  6. ๐Ÿ“ข MetaRex lets you publish services for your metadata type.
  7. ๐Ÿ’ฒ You can monetize those services.
  8. ๐Ÿ”Ž MetaRex lets you discover services for unknown metadata types.
  9. ๐Ÿ‘จโ€๐Ÿ‘ฉโ€๐Ÿ‘งโ€๐Ÿ‘ง MetaRex supports all types of metadata including text & binary, timeline & static, frame locked, sample locked & stochastic, single stream & mixed streams of metadata.
  10. โš–๏ธ MetaRex is built on standard tools, standard formats & standard frameworks.
  11. ๐Ÿ˜Ž MetaRex can run in a product, on-premise or globally.

But why is it useful?

Like most things, it comes down to money. Over the last 30 years we’ve figured out how to move video and audio around at reasonable cost, but the metadata always gets lost!

It’s often faster and easier to invent your own metadata than it is to use someone else’s - often because there’s not a good description of what the metadata is. MetaRex makes those descriptions readily available and so reduces the amount of engineering for metadata interchange and introduces a new monetization opportunity between metadata creators and consumers - even if the two companies have never met.

Some examples

Here are some examples from the MetaRex use-case :

 Live
Production
Virtual
Production
Post
Production
DistributionDistribution
ApplicationNewsOn SetQCLocalizationAvails
Common
Interchange
Formats
NewsML
NinJS
arbitrary .xls
Cooke Lens
Arri Lens
Custom Telemetry
qc.ebu.io
Venera xml
custom csv
imf
ttml
webVTT
Movielabs
custom json
custom csv

Today, most systems in each of these segments have to convert between the formats they are likely to see. Each format converter will have its own eclectic mix of behaviors because the code for these convertors is often a custom integration/plugin by the MAM or Playout or Editor vendor. This results in lots of effort being wasted just to interchange metadata between systems in a reliable format.

Transport makes things worse

For each and every data format of the metadata, there is often a custom protocol or signalling mechanism to identify the metadata. You’ll often find that a simple piece of metadata will have different signalling in different file formats or on different networks, despite the fact that IT’S AN IDENTICAL XML.

If you’re not familiar with this problem, just find an engineer who’s implemented an interoperable subtitling transport system, then buy them a drink and ask "Sounds like moving a few bytes per frame is pretty easy, isn’t it?" then sit back and enjoy the show for the next hour or two while they let off steam.

 Live
Production
Virtual
Production
Post
Production
DistributionDistribution
ApplicationNewsOn SetQCLocalizationAvails
Common
Transports
API
NDI
SDI
ST 2110
API
ZiXi
SRT
ST 2110
file xfer
API
embedded
embedded
QT
imf
api
email
carrier pigeon
 To get an idea of how much this costs, multiply the number of products in each segment by the number of protocols and the number of file formats and then take some arbitrary engineering cost for 20 weeks of engineering (8 for sending metadata, 8 for receiving and 4 for ongoing version upgrades and support). I've called this the (very rough) integration factor below and, yes, I've ignored volume of transactions.
Integration Factor36 rif36 rif20 rif30 rif40 rif
 rif = relative integration factor - A rough comparative measure of the metadata integration effort in each segment

There must be a better way!

Compare with MetaRex

The MetaRex framework requires someone to register meta-meta-data that allows a decoding device to say "Aha, there is NinJS in the MetaRex container". It allows a network interface to say "Aha, I know how to put MetaRex containers on my network".

The MetaRex framework also allows transformers to be registered that can perform a canonical conversion from format A to format B (and possibly charge for it), so if your application receives NewsML but needs NinJS then you can write your own or try a 3rd party one BEFORE YOU WRITE CODE.

The integration factor for MetaRex is simply the number of base formats & networks added together rather than being multiplied together and then multiplied by all the implementations.

 Live
Production
Virtual
Production
Post
Production
DistributionDistribution
ApplicationNewsOn SetQCLocalizationAvails
Today's
Workflows
36 rif36 rif20 rif30 rif40 rif
Tomorrow With
MetaRex
7 rif7 rif6 rif6 rif7 rif

I’ve tweaked the non-MetaRex number to compensate for the range of products available in each segment.

What I’ve not included in the numbers is how MetaRex can give an almost GitHub-like robustness to versioning and updating metadata flow. That’s for a future study.

So who benefits?

Everyone Benefits!

MetaRex enables a generic metadata supply chain with the rigour that we

are starting to develop with video and audio.

  • Content Creators can now track and convert metadata from camera to screen
  • Product Vendors can now integrate more metadata types for less initial and significantly less engineering costs
  • Data Warehousing can benefit by integrating disparate timeline metadata with the same ease as static metadata
  • Metadata Producers can benefit with greater access to more networks and file formats with no extra effort
  • Metadata Consumers can benefit by simultaneously managing and using multiple transformed variants of source metadata without increasing costs
  • The Industry can benefit by spending the money saved by MetaRex on creating rich, metadata driven workflows instead of just leaving the metadata on the floor
  • The planet might benefit if we spent microWatts of energy remembering the source metadata rather than spending kiloWatts of energy guessing what was lost by using an AI algorithm

The MetaRex project needs you! Please become a backer - We’re offering a 10% discount until April 30th 2024. We will happily shout your company’s name across the globe if you can help us get the project over the finish line.

Don’t forget to come and see us at NAB 2024 on the Signiant booth W.1931