PubForge Blog

(This post began life as an email thread, but maybe needs to be more public so here it is.  Edited and expanded for obsessive clarity…)

It strikes me as somewhat simple (OK maybe not exactly simple) to develop a Drupal-based CMS with enough commonly-needed features for public radio/TV stations.  You’d have your pre-built data types, skinable templates, forms, and possibly a set of pre-defined roles and workflows.  All nicely documented etc.

But what we all really want is a system that knows about media files.  You could upload (or link to) a media object, and the CMS would extract its available metadata.  The system would then save that metadata in its database for processing and display in various ways.  On web pages where media is published, the system would display its media type, length, bitrate, framerate, whatever.  Then of course we’d be adding by hand other metadata like title, subject, author, keywords, description, etc as we add media content to the website.  Ideally, the system would be able to automatically read ID3 tags, MXF, and EXIF metadata for both technical and descriptive information.  The idea is to automate the capturing of metadata as much as possible.

For web pages, we’d probably want to display mostly descriptive metadata, and not things like sampling rate, bit depth, color format, etc.

But for RSS feeds we need some of that technical metadata like filesize and mimetype.

And here’s the good part: If we capture enough information about our media objects, we can easily express it as “shareable metadata” via PBCore-compliant XML, and other standard schema.  So the CMS becomes a powerful tool for creating a large index of public media.  We can then write applications to search that index at a very fine level of detail.  Think Technorarti only focused entirely on media objects expressed as detailed XML records.

At WILL we currently catalog media objects (as I call them) using our CMS, but there’s no automatic extraction of anything.  We have to key in all the data.  But once that’s done, the output looks like this:

http://will.illinois.edu/metadata/pbcore/pf2008-04-17-a

Seems to me this is the beginning of a system-wide super API that doesn’t depend on any central organization, and is truly open source.

Existing open source PHP functions for automated metadata extraction could be integrated in a Drupal-like CMS.  The PHP ID3 function allows for reading and manipulating ID3 tags; the PHP Exif Functions can extract all kinds of metadata from JPEGs and TIFFs.  Similar functions may already exist for video files.

If we have a CMS that understands how to read existing metadata from the digital objects we feed it, we’re half-way to building an online digital asset management system.  More on that in Part Two…

Jack Brighton

Under the category of be careful what you ask for…we created a survey to determine how public radio/TV stations are maintaining their websites, their web staffing levels, technologies, and needs. We wanted to find out about the use of open source tools, and maybe identify development projects that would help people throughout the system.

The survey remains open, and we hope lots more people fill it out.

Meanwhile, here’s a quick summary of the stuff that really caught our notice. Made our hair stand on end. One of us cried. Here’s the scoop:

1) We have 64 survey responses from public TV/radio stations as of September 4, 2008.
2) 32 of those stations have no full-time staff dedicated to website creation and management.

Ouch.

3) 18 stations have at least one part-time web staff; 10 have two part-timers.
4) Only 15 stations have a single full-time staff devoted to web development. So that’s what? Less than 25 percent of the respondents.

That’s crazy.

5) Eight responding stations have two full-time web staff; four respondents have three to five full-time web staff.

The picture thus far would include about 28 stations with one or more full-time web staff, and 32 stations with no full-time web staff. (The numbers are a bit odd due to the survey design, but that’s an accurate-enough count.) Some of the 32 stations don’t even have part-time web staff. I hate to go out on a limb here, but I think that reveals a significant deficit in web staff resources at half the stations responding. But wait, there’s more.

6) Only 37 respondents out of 64 have staff skilled in web design, and only 29 have skills in web standards/CSS.

We’ll resist commenting on the difference between skills in web design versus web standards. The numbers in either case are way too low.

Look at it like this: If your station operates a transmitter, odds are better than 50 percent that you have an engineer or three who knows how to make the signal go from microphones and cameras through mixers and switchers and out to the air so people can receive it. Metaphorically speaking, a web server is like a transmitter but rather than mics and cameras we have websites and RSS feeds. Our station websites are now mission-critical: they play an increasing role in delivering content, engaging audiences, fundraising, and delivering on our promise to enlighten communities and inform democracy. So far this survey shows that at least half of us don’t have a single web engineer.

So NPR has rolled out a wonderful API, and soon social networking features and tools. Those of us who understand how to use an API etc can take advantage of these cool new things and make our sites even better. The problem is, too many of us don’t have that understanding.

But let’s not take that as a criticism of stations without adequate resources and skills. In building public media on the web, we’re in this together. That’s why we began this assessment: to find key areas where we can collaborate, share knowledge, and develop best practices.

It’s just that the contrast between the haves and the have-nots is so glaring, and it seems relevant to highlight for further discussion. After all the CPB charter includes the following:

“The mission of CPB is to facilitate the development of, and ensure universal access to, non-commercial high-quality programming and telecommunications services…
…Whether through direct services such as assistance in management practices and access to new technologies…”

Points of Community Leverage

The survey also shows wide use of and interest in open source solutions, and a desire for collaboration. A growing number of stations are using Content Management Systems, including Drupal, Zope (OK it’s really a framework), Joomla, ExpressionEngine, and Public Media Manager. Others are using home-grown CMSs built with open source elements and scripting languages. Open source tools and systems are used by almost half (31) of the respondents.

21 of the respondents expressed concerns about using open source based on…their lack of staff skills and support resources. The same barrier rears its unlucky head once again.

The good news is the vast majority of respondents are interested in learning more, and collaborating around open source projects that would serve the needs of all stations. A solid majority of respondents say they would travel for training sessions.

You can of course read the full analysis of the survey on this site, and we’ll update it as we get additional responses. Meanwhile, it looks like PubForge has an opportunity for some heavy lifting. We’d like to make a big difference for both the haves and the have-nots. And you know what they say about many hands making light work. Let’s use this as inspiration and get busy.

As I’ve been reading over the PubForge Open Source Collaboration Survey results (over and over again), I’ve begun to see between the lines a bit. I don’t mean I’m getting dizzy, although that could be true as well. No, I mean that as is often the case, I find myself beginning to understand the hidden meaning and unspoken concerns that always seem to poke their heads up gradually as you review data over time.

The head that I’m seeing poking it’s head up at the moment has a lot to do with sustainability for public media web operations. We talk a lot about this subject at conferences and on webinars. But I wonder, as we have those discussions, have we really given thought to what it means and why we should be concerned. It seems to me we can’t sustain ourselves by trying to figure out how to do the web better than we are doing it now.

Let me get right to the point: Sustainability does not mean putting more time and money into the web or web projects or figuring out how things work on the web. It means putting more time and money into the people who run the web and web projects, so they can develop policies, systems and processes that will make it possible for our projects to sustain themselves.

Two simple facts pointed me in this direction of thought.

First, that the single loudest cry for help found in our survey results was the repeated, “heellp meee…” that resounded when we asked everyone, “what do you want to do and why haven’t you done it yet?” We saw a lot of answers along the lines of “no tech staff” and “no support” and “limited expertise”. In short, we’d love to embark upon the greatest web project of all time but we’ve no idea how to do it or how to keep it going over time.

Second, the number of great ideas that came in through this survey was amazing. So far we’ve seen an amazing outpouring of what amounts to contributions to collaborative projects, and we haven’t even created a mechanism for doing so yet. There are a lot of great people out there in public media who have all kinds of ideas about how to make our services better, richer and more competitive.

I take these two facts and I add them to something else I know about having been a part of the Public Media Metrics project for a while now: Most station websites are lucky if they see the same visitor come back to their website more than once each month. Repeat traffic rates for public media station websites is pretty abysmal, and this suggests that creating any kind of model for ongoing web presence sustainability is laughable.

What’s my result after putting it all together? Let’s see…hmmm… for the geek audience here, I think it’s something like this:


$sustainable     = false; // ASSUME THE WORST AT THE START AND TEST IT OUT LATER
$good_ideas      = 1000000;
$visitor_loyalty = count($visits_per_visitor[$month]);
$have_help       = count($web_staff);
$have_support    = getSupportFromVisitors($visitor_loyalty);

foreach ($good_ideas) {


if ($have_help == true && $have_support > $not_much) { // $not_much IS NOT DEFINED

$sustainable++; // BONUS!!!

}
else $sustainable--; // BOGUS!!!

} // LOTS OF THINGS STAND IN THE WAY OF GETTING HERE...

Sorry - for those who don’t eat code for breakfast, this mostly breaks down to, “if we don’t have support from visitors, and we don’t have staff, we can have a million good ideas and it still don’t amount to spit.”


// === END SARCASM === //


One of our survey respondents made the point that we should ask CPB for help. At first glance I thought, CPB has helped quite a bit, so why are we still here? Is it because we don’t know how to make good things happen when we get money? Is it because we are wasting it when we get it? I don’t think either one of these things is true. In fact, I think that there are lots of folks out there in public media who have done a lot of good things online with the money that’s been granted by CPB and other organizations. We are a productive and thrifty bunch.

It seems to me that the real issue when it comes to sustainability is all about infrastructure. It’s all well and good to grant $10,000 (or way more) to spend on the coolest web project that ever was. But how many stations would have the expertise and services they need to do anything with that kind of money? There are many, many stations out there who would love to have that kind of a budget for use online. But I’ll bet they’d like to spend it on salaries, not on “projects”. I may be shooting myself in the foot here as I am a consultant myself, but what good does it do to send a heap of dough to a station to spend on hiring consultants who will build a service or feature for them that, once those consultants are done and gone, the station won’t know what to do with over time? Should that station hire the consultants again to come back and fix it/use it/upgrade it/update it/expand it… you get my drift.

We wouldn’t put a station on the air without having programs to air, a tower to transmit, and staff to mind the programs and the tower that sends it out, BOTH. Likewise, we cannot expect to put a quality online service into the ether without having content, a website, and staff to mind the content and the website that sends it out, BOTH.

PubForge was formed of this notion that we are under-resourced and that together we can help each other with problems that would normally be insurmountable on our own. The very concept of PubForge is to build a community of virtual staff who can materialize out of thin air when the need arises, for whatever station might be in need at any given time. But the one thing that PubForge can’t do is provide sustainability for each individual station. All the great collective ideas and collaborative systems and community projects in the world will not change the fact that without a human being or two sitting at the keyboard punching code or publishing content, there is no way to sustain the web operation.

So what is necessary for sustainability? We absolutely MUST invest in more than just projects. We need at least one, if not two full time bodies in each station that hopes to have a web service. Granted, once those bodies are in place they’ve got their work cut out for them. There’s no shortage of problems to solve for the full time web pro in public media, starting with building visitor loyalty and reaching all the way to monetization. Those are huge ailments in need of serious triage, and any emergency team attempting to heal that patient will be pretty busy for a while. Thing is, what would you do if you had that patient right there in your station, but no emergency team attending?

I can think of at least 33 stations right off the top of my head who don’t even need to guess at the answer.