A few notes on bbPress

I have been spending a lot of time off and on the past week creating a bbPress (message boards, or “forum software,” or “bulletin board”) theme for eMusic. Theming is fun because you get to touch almost every feature of the product, bbPress and/or WordPress. In the midst of this, I’ve been discovering how bbPress does things and making some modifications along the way. Those mods may only exist in eMusic and never make it into bbPress, but I have shared ideas with JJJ so I figured I would share them here as well for anyone to check out and supply feedback.

Queries within Queries with Queries

bbPress abstracts almost everything. For most WordPress-y things, there is a bbPress-y thing. One thing that will immediately confuse and potentially wreak havoc is the way “topics” are queried. Because the default WordPress query is the page or custom post type you are on, the topics for a forum are queried using a second mechanism: bbp_has_topics( ), which upon success, returns a WP_Query object.

That’s all well and good, but at any given time, there is only 1 bbPress query and no hardened reference to the original query. If you are using the default theme or just modifying its presentation, you probably don’t care. If you are implementing a design that has multiple “topics” queries, you are kinda up shit’s creek unless you roll your own code. The template tags / functions provided by bbPress will work to an extent, but they will also shift bbPress’s entire context every time you call bbp_has_topics( ).

The solution espoused to me was to roll my own WP_Query objects to replicate what bbPress is doing in the background, but I don’t want to do that. I want to use bbPress functions and fix whatever context is set along the way when necessary. WordPress maintains state and stores the main query by using $wp_the_query and $wp_query, for the main query and then the current query, respectively. bbPress clobbers $bbp->topic_query every time bbp_has_topics( ) is run. bbPress is treating bbp_has_topics( ) like query_posts( ) in WordPress, not like new WP_Query( ). Ask Nacin how he feels about query_posts( ).

bbPress makes an attempt to provide query context by providing the function bbp_set_query_name( $name )

Guess what it does internally… basically nothing. So I wanted to fix this, here’s some code I am using:

global $emusic_the_bbp_query;
$emusic_the_bbp_query = array();
function emusic_set_bbp_query( $name = 'main', $params = array() ) {
    global $emusic_the_bbp_query;
    $bbp = bbpress();

    bbp_set_query_name( $name );

    if ( !empty( $bbp->topic_query ) && empty( $emusic_the_bbp_query ) )
        $emusic_the_bbp_query['main'] = $bbp->topic_query;

    if ( !empty( $name ) && isset( $emusic_the_bbp_query[$name] ) ) {
        $bbp->topic_query = $emusic_the_bbp_query[$name];
        return $bbp->topic_query;
    }

    if ( !empty( $params ) ) {
        bbp_has_topics( $params );
        $emusic_the_bbp_query[$name] = $bbp->topic_query;
        return $bbp->topic_query;
    }
}

function emusic_reset_bbp_query() {
    global $emusic_the_bbp_query;
    $bbp = bbpress();
    $bbp->topic_query = $emusic_the_bbp_query['main'];
    bbp_set_query_name();
}

My code does a few things:

  • Sets $emusic_the_bbp_query whenever you start to change context the first time
  • Always allows you to retrieve the default context when you are done with a sub-query for topics
  • Non-persistently caches topic queries by name

So where’s my use case? I want to show a “snapshot” of your subscribed-to topics, favorite topics, popular topics, and super-sticky topics in the sidebar on every page, and I want to reuse theme code (template parts) to do so. Imagine calling query_posts( ) 5 times in a template on the WordPress side…

Could I have accomplished this with WP_Query? Yeah, but… I want to use the bbPress stuff. So here’s what I do:

<?php if ( is_user_logged_in() ):   ?>
<div class="meta-block">
    <h3>Your Topics</h3>
    <span class="double-line-narrow"></span>
    <?php
    if ( bbp_is_subscriptions_active() ) : ?>
        <h4 class="sub-head">
            Subscribed
            <a class="aux" href="<?php bbp_user_profile_url( get_current_user_id() ) ?>">view all</a>
        </h4>
        <span class="double-line-narrow"></span>
        <?php
        $subscriptions = bbp_get_user_subscribed_topic_ids( get_current_user_id() );
        if ( !empty( $subscriptions ) ):
            emusic_set_bbp_query( 'bbp_user_profile_subscriptions', array( 'post__in' => $subscriptions, 'posts_per_page' => 3 ) );

            while ( bbp_topics() ) : bbp_the_topic();

                bbp_get_template_part( 'loop' , 'single-sidebar-topic' );

            endwhile;
        else:
            printf( '<p>%s</p>', __( 'You haven't subscribed to any posts.' ) );
        endif;

        emusic_reset_bbp_query();

    endif; ?>

    <h4 class="sub-head">
        Favorites
        <a class="aux" href="<?php bbp_user_profile_url( get_current_user_id() ) ?>">view all</a>
    </h4>
    <span class="double-line-narrow"></span>
    <?php
    $favorites = bbp_get_user_favorites_topic_ids( get_current_user_id() );
    if ( !empty( $favorites ) ):
        emusic_set_bbp_query( 'bbp_user_profile_favorites', array( 'post__in' => $favorites, 'posts_per_page' => 3 ) );

        while ( bbp_topics() ) : bbp_the_topic();

            bbp_get_template_part( 'loop' , 'single-sidebar-topic' );

        endwhile;
    else:
        printf( '<p>%s</p>', __( 'You haven't favorited any posts.' ) );
    endif;

    emusic_reset_bbp_query();
?>
</div>
<?php endif ?>

<?php
$super_stickies = get_option( '_bbp_super_sticky_topics', array() );
if ( !empty( $super_stickies ) ): ?>
    <div class="meta-block">
        <h3>Featured Discussions</h3>
        <span class="double-line-narrow"></span>
        <?php
        emusic_set_bbp_query( 'bbp_super_stickies', array( 'post__in' => $super_stickies, 'posts_per_page' => 4 ) );

        while ( bbp_topics() ) : bbp_the_topic();

            bbp_get_template_part( 'loop' , 'single-sidebar-topic' );

        endwhile;
        ?>
    </div>
<?php endif ?>

Meta Queries like whoa

First of all, JJJ has done an awesome job making bbPress a plugin, and it is super sweet how seamlessly it integrates with everything else in WordPress. The two areas I would like to help make improvements in are 1) cache and non-persistent cache 2) Meta Query performance in WordPress as a whole. bbPress makes Meta Queries like (holy shit) whoa. For 99% of installs, who cares. For us, my current dataset is already 100s of 1000s of posts and 2-3 million rows of postmeta. A meta query basically says:

  • I need stuff from the posts table
  • I need stuff joined from the postmeta table
  • I have indexed columns in both
  • Fuck that noise, let’s join on an unindexable LONGTEXT column and pray for mercy

I wish meta_query never existed in WordPress. But it does, and bbPress sings its song throughout. I have a few ideas for reducing number of queries made and also splitting the query into 2, so that PHP can compare integers instead of trying to sort them as text in MySQL. I have done a lot of performance testing on meta_queries, and I actually ditched them in a few places for WP_Query because they do not scale out of the box.

But I’m not just gonna whine about it, I’m gonna try to contribute and I’ll report back when I do.

Best Albums of 2012 (So Far)

The Best


fun., Some Nights

Unequivocally, my favorite album this year so far. I have listened to it more times than I can count, and I still get goosebumps like it’s Justin Bieber and I’m a 13 year old girl from the Houston suburbs.

If they hadn’t made Aim and Ignite, I think they would be viewed as some mechanized pop creation, but it’s clear Nate Ruess is just a total badass.


Porcelain Raft, Strange Weekend

Completely out of nowhere, this record kicked my ass. The “strange weekend” in question is supposedly the one in which the hurricane was supposed to hit NYC in August of 2011, but obviously didn’t. That weekend was my birthday weekend, and I spent Friday night in Williamsburg, Brooklyn. So did Mauro Remiddi – he had recently moved from Rome via a brief stint in London. COINICIDENCE?


Metric, Synthetica

I was actually getting tired of Metric. I saw them at Terminal 5 in 2009 and 2010 and they played the exact same show – which was awesome, but kinda lame of them. Yeah, we get it, Fantasies was an awesome record.

This new record rules. I am excited to see them perform full blast live. Which is great – because I took a leap of faith buying tix to their show at Radio City Music Hall in September (4 months ahead of time, for $150 – but screw you, I got FRONT ROW).


The Walkmen, Heaven

I used to *dislike* The Walkmen, and then they released Lisbon a few years ago, which if you haven’t heard, is INSANELY good. They opened up for Grizzly Bear at the Beach at Governor’s Island, and I basically ignored them, because I am an idiot.

I went to their headlining show at Terminal 5 and was blown away. I can already hear how awesome these tunes are gonna be live – Remember Remmmmmmmemmber!


Bruce Springsteen, Wrecking Ball

I would never have given this album a chance if I hadn’t seen Bruce and the E Street Band absolutely destroy the Meadowlands a few months ago. Chris and I saw Bruce at Giants Stadium (the old one) before they tore it down and rebuilt the new one.

At that show (the one where he played 100% of Darkness on the Edge of Town), before the band came out, Bruce came out and played a new song by himself: “Wrecking Ball.” The whole album is good. Granted, it may take seeing him live to respect what an absolute musical giant he is, but I have listened to this record basically once a day for 3 months.

The Rest

Escort, Escort
Daughn Gibson, All Hell
White Rabbits, Milk Famous
Silversun Pickups, Neck of the Woods
Sharon Van Etten, Tramp
Beach House, Bloom
of Montreal, Paralytic Stalks
Japandroids, Celebration Rock
Heartless Bastards, Arrow
Bear in Heaven, I Love You, It’s Cool

Listen To This: “Synthetica” by Metric

I LOVE Emily Haines. She is my favorite part of Broken Social Scene. I saw Metric give 2 amazing performances at Terminal 5 when they were supporting their last record, Fantasies. I have front row tickets to see them at Radio City Music Hall in September.

Their new record dropped, Synthetica, dropped today. It’s awesome:

WordCamp NYC 2012: “Cloud, Cache, and Configs”

Here are the slides from my talk today:

I spoke for 40 minutes to a room full of people that had no idea what I was saying. Seriously.

Have any of you made a plugin before? Silence / crickets. Cool, well let me dive into scaling HTTP parallelization for 15 minutes…

Installing libmemcached on CentOS

I recently updated the Memcached WP Object Cache plugin to use the Memcached PHP extension – it currently uses the Memcache extension. My plugin is called Memcached Redux, and it’s delicious. The Memcached PHP extension implements methods from libmemcached, the C / C++ Memcached library.

Installing the extension on MacPorts is beyond easy:

port install php5-memached

Installing on Amazon EC2 instance running CentOS 5 is WAY more esoteric. I spent about 2 hours mucking around until I figured it out, so here it is (this assumes you are already using memcached and have libevent, libzlib, etc installed):

cd /etc/yum.repos.d/
wget http://rpms.famillecollet.com/remi-enterprise.repo
wget http://syslogserver.googlecode.com/files/epel-release-5-3.noarch.rpm
wget http://rpms.famillecollet.com/enterprise/remi-release-5.rpm
rpm -Uvh remi-release-5*.rpm epel-release-5*.rpm
yum --enablerepo=remi install libmemcached*
pecl install memcached

vi /etc/php.ini

// (under Dynamic Extensions) Add "extension=memcached.so":

extension=apc.so
extension=http.so
extension=memcache.so
extension=memcached.so

Servers have to be restarted – this has be done before code is deployed or the class won’t exist, which will cause a fatal error.

This process looks harmless, but I had to read 3.7 million blog posts before I found the right path forward.

Memcached Redux

I’ve been reading about Couchbase, and I knew it was compatible with Memcached out of the box. One of the features I wanted to start using was Memcached::getMulti and Memcached::setMulti. I knew the Memcached WP Object Cache plugin had a get_multi method, but I didn’t know what it did or how it did it. Turns out, it doesn’t implement Memcached::getMulti.

When I looked under the hood, I realized that the Memcache extension is loads different than the Memcached extension in PHP. Memcached has the getMulti method, Memcache does not. So I set out to change this: I have altered the famed Memcached plugin to actually use the Memcached class.

Because I did this, you can now use methods like this:

wp_cache_get_multi( array(
	array( 'key', 'group' ),
	array( 'key', '' ),
	array( 'key', 'group' ),
	'key'
) );

wp_cache_set_multi( array(
	array( 'key', 'data', 'group' ),
	array( 'key', 'data' )
) );

Rather than making many calls to grab data, especially related data, you can grab it in one hop.

I’m gonna keep working on the plugin, and drop a line if you install it and have comments / concerns.

The Plugin: Memcached Redux

Why Spotify Sucks

I just watched the Session Highlights of Spotify CEO Daniel Ek and investor/adviser Sean Parker’s talk with Walt Mossberg at the All Things Digital Conference, and it confirmed what I have thought for a long time: Spotify sucks, is bad for artists, and hopefully does not succeed. I am not saying this because I have bias (I work for eMusic). I am saying this because I am a musician, I know a ton of musicians, I play in a band, I have toured with my band, and I have seen behind the curtain of how the music industry works. I know how little money there is to go around it, and I can give you some examples.

Return of the Era of the Single?

One of the most irritating things I hear is that “people don’t want full albums, they only want singles.” People generally will say “albums only have one or two songs that are worth hearing, the rest is filler.” So, that’s a giant lie. Yes, I’m sure the one Chris Brown song you heard at the mall is the only good song on his record. But if you’re buying a Chris Brown record and going to the mall, you aren’t the kind of person that is going to deeply engage with a digital music service anyways. You probably also have bad taste. If you are a real music fan, you buy records because you like the artist. You want to hear the entire record.

Artists don’t go into the studio to record one good song and then load up the rest of the record with filler. I’m sure there are plenty of pop factory records that do, but in that case, the label is probably shooting to moon and trying to launch the new teen sensation or something. *Real* artists are trying to create something great. You would never get a Radiohead record and skip through the tracks until you found the hit. You listen to the entire record, know every song, and view the work as a whole, as a period in the band’s life.

Saying that people only want to consume bands and artists in a hyper-unaware, ADD fashion is insulting and a piece of misdirection. “Album sales are down, accept that people might pay for one song only and be happy about it!” No. There is nothing better than a great album by a great band. And there are many, new ones arriving weekly.

The simple facts are: an artist will make more money from the purchases of their album, digital or otherwise, than they will from the sum of streaming royalties. Let’s say an album sells for $10 on iTunes, the label gets $7, and the artist gets $2 of that. If the albums sells 100,000 copies – which in this day and age, is what a semi-famous indie will sell – the artist gets about $200,000. Assume the artist is a band with 4 people: $50,000 each (let’s make it $37,000 after taxes). If you’re from my hometown where the median income is in the mid-$20K range, congrats: you’re in the middle class. If you live in Brooklyn (probably Bed Stuy or Flatbush on that money), you make less than a 23 year old on their first job post-college.

So that money doesn’t look very good, right? Let’s look at another scenario. YOU’RE the label. You made the record in your bedroom, but it sounds great, and Pitchfork wrote about you. 100,000 albums is a dream, you sold 20,000 albums – huge numbers for a DIY-er. You make $140,000 / take off taxes / divide by 4. You’re poor.

Artists Have Other Revenue Streams, Right?

Um, not really…. I mean, there is touring. Let’s break down touring. Assume you are in my band, and you want to go on tour for 2 weeks (we’ve done this, a bunch of times). Before you even leave home, the vehicle rental is a fixed cost, about $3,000 – could be cheaper depending where you tour, but ultimately, it’s expensive. If your only income is your band, this could be a non-starter. $750 per man from someone making $1,000 a month is a lot of money. Remember, the money made from an album doesn’t come in one lump sum, and the royalties are cumulative for an entire 1-2 years, and money is probably only paid every quarter, if that often. And the money is only a reality if 1) you made an awesome record and 2) people buy it. Not just “people,” but every person who likes the album. You can see where our numbers fall apart again.

So not only do we have to pay about $3,000 for our van, we need to put gas in it every day. Not sure if you’re aware how expensive gas is these days, but speculation by banks into the commodities and futures markets have caused things like gas to shoot up into the holy shit expensive range. When you go to fill up your gas tank, expect to drop $100 every time. Let’s assume you fill up 2/3 of the days you are on tour (10 days). There’s another $1000 in fixed costs.

You might be asking, why not play venues that are closer together? You would save money! That would be awesome, but that’s not gonna work. Every state in America has a finite number of towns in it that are even remotely interesting. Try this: name 6 towns in Virginia that have music venues that support your style of music. Ok, now get booked in all 6 towns. You can’t because: you may have zero draw (number people willing to come see and pay money for it), the venue is already booked, the promoter might not like your music.

But let’s shoot the moon. Let’s say you booked all 6 venues back to back. Since you’re on a 2 week tour, let’s assume you’re playing every night. For one hour. Monday-Saturday, 6 nights in a row in Virginia. Since you’re new to the area, cover is $5 and you keep everything after the first 20 people pay. For four people in your band to split $100 a night, you need to have 40 paying customers every night for 6 nights. That means: to cover your gas for Tuesday and zero dollars left to split, you need to bring 40 people out to a music venue in Virginia, that may or may not even know who you are and have them plop down $5 to do so. You’re also trying to do this at a venue in a potentially backwoods town, and the venue might have 5 bands a night, 7 days a week.

This is not meant to be a horror story, this is reality. It is insanely hard to make money touring at a DIY level. But what you’re probably thinking is: yeah, but larger acts make a fucking killing, it’s just your shitty band that can’t make any money. Fair, but let’s break that down. Aerosmith? Yes, private jets, made $20M for 10 shows in Rio recently, boatloads of cash. Springsteen could sell out 20 nights in a row at Giants Stadium. But the number of people who can even come close to doing anything like that is very very small.

I know a band (they’ll remain nameless) that was recently in Pollstar’s Top 50 grossing acts. The band grosses about $1M a year from touring. They make guarantees at venues. A talent buyer calls up their booker and says he’ll give them $X to play at Y venue. The band thinks it over and might accept. They have a consistent draw in enough markets to command a decent guarantee. The band also sells a lot of merch. Because they are a jammy / folky kind of band, they don’t sell a lot of records. So, touring is their main source of income. Band has a booker, a manager, a crew, and 5 band members. My friend is in the band. What is his cut of that million dollar pie? $32,000 a year. $10K less than I made at my first job in NYC. And they’re a SUCCESS story!

Their tour bus runs about $700 a day. They have to add some production value to their show, so they have a crew. Manager takes his cut off the top, and they have to take out money for taxes. My friend assures me that NO ONE is making the money you think they are making from touring. A lot of major pop acts will release a single from a new album, and if it doesn’t blow up like they expect, they’ll cancel their stadium / arena tour, knowing how expensive it is to not have consistent sellouts (Kelly Clarkson and Christina Aguilera both cancelled tours a few years ago, rather than scale back).

So where are those other revenue streams?

Spotify pays about $0.0004 per stream. Let’s say you write an amazing pop tune, the internet goes crazy, and Spotify’s users stream that song One Million Times. $400. You and your 3 bandmates are taking home a cool $100 a piece. Having your song streamed a million times means your band has some level of fame. And your reward for blanketing Spotify with your single is 400 fucking dollars. Let’s assume my math is WAY off and Spotify actually pays a penny ($0.01) per stream. Congrats on your cumulative payout of $2500 per man. If you are super broke, you’re happy, for a month.

Of course, let’s realize that almost no one generates 1 million streams. So almost no one is making that much money, which isn’t even a lot of money.

Artists Get Exposure?

I have heard SO MANY people defend Spotify and say “but artists get exposure.” How? How does generating a $0.0004 payout to an artist qualify as anything other than “I just got to hear music on demand for free.” That artist probably spent $2000-10000 making their record. So those million streams are just paying them back for that (if they get the huge royalty rate of one fucking penny per stream, which is absurdly high). They then hope to translate those listens into paying customers at their shows.

Here is where the logic falls apart for me, because none of the people who say that (“artists get exposure”) are in a band, have toured, and have had to generate an audience of paying customers. It is very very hard to get all of your FRIENDS to come to a show, much less strangers in a area where you are not from and have no roots. Saying that streams translate into concert attendance makes the false assumption that everyone who listens to music sees all of those bands in concert every chance they get.

MOST people don’t ever see live music. MOST people don’t go out on Tuesday nights. Some markets work different than others. In NYC, you might have people come out on a Tuesday night and see you. In some college markets, the same. But even in rowdy college towns, you can’t always rely on a huge turnout on weeknights, even from those people who enjoy your music.

To make the argument that somehow Spotify changes this is ludicrous. Otherwise, what does “exposure” mean? Does it mean this listener is now going to spend top dollar for your goods? They were just of Spotify to SAVE MONEY, and NOT PAY full price, or at all, for your music.

There is no added exposure, there’s $0.0004 to you and your buddies.

Music is not Free

I don’t care how this sounds – Lars Ulrich was ahead of his time. Napster somehow made everyone believe that music is free and it is Your Right to be able to download the shit out of it, at will, for eternity. In that scenario, musicians are basically expected to live in abject poverty. Yes, you’ll have your manufactured stars that appear to have it all, and the few that break through actually will. But in society these days, music and movies and art are seen by the general public as not a product, but as an unalienable right that all people have to download and then vote on the life or death of the artists who created them.

At the end of last year, I was really into The War on Drugs lastest record. I actually see all the bands I respect live, so I was at their sold-out show at The Bowery Ballroom. I hung around after to have some beers and maybe meet the band, since I noticed they were packing up their merch tables themselves. Standing there, I overheard the lead singer’s friend ask the band where they were crashing that night (the band is from Philly). Lead singer: “The Red Roof Inn in Secaucus, my dad got us 2 rooms.”

LCD Soundsystem played their final show at Madison Square Garden last year, rounding off a year of a tour or 2 at some of the biggest venues and festivals across the world. James Murphy appears to make a bunch of dough, but I know someone who knows Nancy Whang (maybe the 2nd most recognizable person in the band), and she’s close to 40 years old and lives with 2 roommates. I know other members of the band do a lot of substitute teaching.

When I saw the Dirty Projectors play the 4th of 4 sold-out shows at The Bowery Ballroom a few years ago, they were setting up their own gear because they don’t have a crew, and that was at the height of their popularity – The Roots and David Byrne joined them on stage that night.

When Sharon Van Etten made her record last year, she actually didn’t live anywhere – she was crashing with different friends throughout. And these are the SUCCESS stories. These are some of the most revered artists of present day. Magazine covers, indie label contracts, the occasional video, Take Away Shows, sold out NYC shows. And they’re just scraping by. Spotify’s not saving them, and won’t.

Imagine what Spotify’s not doing for artists that aren’t at the top of the game.