I got an email from James Brausch with the Testing newsletter attached shortly after writing the last post about the free issue (thank you James, and thank you to the intern or contractor who was on top of it so quickly).

It's basically 15 pages of test data from his InternTask.com site, which is the "sales" page for his free intern program.

The data is all pulled from Muvar, his multivariate testing software. It covers a number of variables that he is testing on InternTask.com.

There were several interesting pieces of data I got from it. First, nothingness often beats all other variables. For example, removing a particular paragraph from the page may produce better results than any variations of the paragraph did.

Second, I got some ideas for things I can test on my own sales pages (a number of which are running Muvar) that I hadn't thought of before.

Lastly, I found it interesting how he values a signup on this site, given that it is a no-cost intern program and he doesn't directly make any money from it. This isn't exactly a testing-related item, but it was still something I found useful.

Is the Testing newsletter worthwhile?

I took another look at the sales page for it and from the description there it sounds like each issue is going to cover nothing but data from one of James' websites.

I may be wrong, but my expectation is that James would be using similar tests on most of his websites. I don't think that there would be enough new data from one month to another to justify the price.

Plus, testing is not something that can be completely generalized. What works on one site doesn't necessarily work on another. You can use the results from one site to make some basic assumptions for another one, but ideally you would still want to test the losing variable from the first site to make sure they don't perform differently on the second. The information in the newsletter is helpful for getting ideas to test, but I wouldn't use it verbatim without running my own tests on it.

Personally, I don't think I would get $300 worth of value out of the newsletter. I already use Muvar myself and have a pretty good handle on how it works, so I can generate my own results pretty quickly - and they're for my site so I know they're applicable.

If you have trouble coming up with ideas for variables to test on your sites or you want to "watch over someone's shoulder" to see how they implement testing, the newsletter might be worth it for you.

If there is more to it than purely test results, I might reconsider whether it's worth the money for me. If there is more to it, perhaps someone could leave a comment with that information.