Tuesday, September 16, 2008

3rd Party Marketing Tracking Systems (Web Bugs, ick)

A smart developer I once worked with, Andres, called this a web bug system. When he built it back in 2000 (I think?), it was the first one that I had ever heard about, though likely others were also built around that time. Since that job, I have either had a similar system built or wished I had a web bug serving system at every company I worked for.

Here's the deal: web bugs, transparent gifs, 1x1 tracking pixels, whatever, are nasty as far as I am concerned. If you haven't worked with them before, here's how it goes. Let's say that ABC advertising is sending traffic to my site, but I only have to pay them when a customer purchases something on the site. I could just look in my database and tell them how many people purchased from the ones they referred to me, I could even ping them in real time or send data to an API they built for that purpose. But laziness is much more prevalent than trust, so web bugs are used most of the time instead. I put an image on my checkout/confirmation page with ABC advertising's domain, and then when my customer's browser calls that image, they can count the purchase. They can also look for their cookie on the customer's machine, and tie each conversion to a specific referral. To make things even juicier, they could see and demand payment for any purchaser that they sent in the last 7 days or the last 30 days because they introduced the customer to your site making it more likely they were going to use your services. They could even see customers that they never referred and infer something about those customers based on the fact that they are using your site (just bought a camera? maybe I should market memory sticks to you).

More recently, some of these tracking pixels were being blocked by browser plugins and privacy schemes, so many of these tags have been getting switched from image tags to java script calls. Since the privacy plugins do not know if the java script is there for tracking or actual page functionality, it would be a little riskier for them to block a java script call automatically. Also java script can suck a lot more information out and do a lot more than a simple image call. For instance, some ad networks get the java script call and drop a whole bunch of image tags on the page automatically. By the time you get dozens of different advertisers and networks and some of them piggy-backing dozens of tags onto one java script call, you can end up with a lot of tags calling all over the place, and you can also end up with more than one advertiser claiming responsibility and demanding payment for your referral if you are not careful.

So build yourself a web bug system like that works like this:

- Query string tagging: Tie inbound query string parameters and their values to firing a web bug. For instance, tell Yahoo to use the link www.mydomain.com/widget.html?tid=12345 when they send traffic to your site. tid is short for tracking id, and it tells the web bug system which web bug to fire. So Yahoo will get a RightMedia web bug for any traffic they refer, but their web bug or javascript will not fire when someone else comes to the site with a tid=54321 query string in there.

- General regular expression matching: Ideally, your system should not only accept tidd, but be able to look for any query string parameter/value pair that you put into a regular expression matcher of sorts. This is because occasionally you will get publishers that are picky about the URLs they are sending traffic to you with. It may also allow you to minimize the number of query strings on the inbound url, which is more consumer friendly.

- Trigger events: Create a set of actions that can trigger firing the web bugs. So in my example above, I talked about the checkout page having a web bug on it. You might also want one for user registration, email newsletter sign up, whatever you are going to pay marketers for. Some marketers and partners may want one on the landing page so they can count total referrals, and some may want a tag on all of your pages so they can monitor their referral's use of your site (yuck!!!! but it does happen). In the end, you may end up with 3 or 4 events, which combined with a query string parameter/value pair on the inbound, cause you to fire a web bug.

- Persistence: If you want to get advanced, use your own cookie to look for and credit prior referrals. This will make publishers that want to get paid for all conversions in the next 30 days happy. So here's how it goes, assign a web bug stickiness when referral shows up the first time. For example, all hotmail referrals get a web bug on checkout page from this time through the next X days. Maybe Yahoo referrals are sticky for 30 days and Google referrals are only sticky for 7 days. Regardless, if the cookie with the sticky visit shows up again, you would still fire the web bug for them, so you have to start looking at the cookie id on the way in for this as well as checking the query string. Not too bad really.

- Precedence: Complication arises if you want to avoid throwing multiple tags for the same person. What if someone came from Yahoo 24 days ago and Google 2 days ago. Do you fire both? Well, it depends. If you really want to get this right, you would fire any tag which is used for tracking but not payment. But if you are using the tag for payment, you would only fire the tag for the one that sent you a referral most recently, regardless of deadline. You can get really convoluted with precedence rules if you want, but what I just listed is quite enough really.

- Cache busting: one last thing, when the web bug fires, many people want a random non-repeating number thrown into the URL somewhere. Sometimes as a query string, sometimes in the directory structure, sometimes more than once in a single URL. They are trying to make sure the browser does not grab a cached image so the browser actually calls the remote server for the image again. So you should have a cache buster variable you can substitute into the web bug call URL or query string at will.

Tags: webbug, tracking, pixel, publisher, referral, gif, transparent
Wednesday June 18, 2008 - 05:25pm (PDT) Permanent Link | 0 Comments

1 comment:

Unknown said...

We understand that we need this but not sure how to go about making it happen - we don't have anyone with JS skills on staff. I'm not even sure we could spec it out properly. Bottom line is we want to avoid duplicate commissions across affiliate networks. What is your best advice on getting help with this?