skippy the bush kangaroo

Wednesday, March 21, 2007

wizard of kos 2: bad news for booman

in our recent (and recommended at boomantribune, frontpaged at myleftwing...thank you all...sniff...we couldn't have done it without the little people) diary the wizard of kos, we tried to prove with real facts that kos is a jerk. well, actually, what we tried to prove was that his purging of his blogroll was objectively a bad thing. we kind of did, too, tho we were pretty much preaching to the choir. can we get an 'amen'?

however, booman brought up the idea of individual dkos user blogrolls on their own individual diaries:

skippy-

markos gave every diarist their own blogroll, thereby immensely increasing the number of bloglinks emanating from his site. i don't know how powerful those links are compared to the homepage, but it would seem he has greatly increased the potential for helping google rankings and taking himself out of the decision making process.

in the interest of fairness, you might mention that, since it seems unlikely his intent was to hurt bloggers by jacking up their links by several orders of magnitude. I, for example, used to have one link from there (two if you include my sig) and now I have 80 or something. Overall, it hurt my traffic, but it probably helped by google ranking.
well, bad news for booman: it not only hurt your traffic, it didn't help your google ranking one iota.

we wrote to bill slawski, the author of the original piece we based our post on, and of the blog seo by the sea - internet marketing and search engine optimization research and services. mr. slawski wrote back:

each individual diary appears to be created on its own unique subdomain at dailykos.com. what that means for purposes of indexing at google is that each is treated as a unique website rather than part of the dailykos site. you can see when you perform a search on google for "dailykos" that in addition to the main dailykos home page, the results are also filled with the diary sites. google normally will only show two results from the same site in their search results page, with the second result being an indented one under the first. since it isn't filtering out pages from the diary sites, it does appear that it is treating them as separate sites.
[ed. note: don't get excited, kids. keep reading]

there is a benefit to each of the diary sites in having links to them from the front page of the main dailykos site, and from other pages of the main dailykos domain. hard to tell if the benefit of links appearing in the main content area of the front page is any different than links appearing in the sidebar.

on each of the diary pages, there is a link labeled "blogroll" that points to a page on the main dailykos site. these blogrolls aren't on the dairy site subdomains, so they aren't separate blogrolls for each of the diary sites, but rather pages that would be considered by google to be part of the main dailykos site. but...

if you look at the robots.txt file for the dailykos.com site, you'll see that the user pages are disallowed for a number of robots (http://www.dailykos.com/robots.txt).

user-agent: googlebot
#disallow: /
disallow: /comments
disallow: /user
disallow: /poll
disallow: /print
disallow: /search

oddly, yahoo's slurp isn't disallowed, nor is msnbot.
[ed. note: go figure!]

the diary owner's blogrolls are being disallowed to googlebot by the robots.txt files. the url structure of the diary owners' blogrolls is like this:

http://www.dailykos.com/user/username/blogroll

because of this line in the robots.txt file - "disallow: /user" - google isn't allowed to visit those diary users' blogroll pages, and index them, and follow the links upon them. in terms of ranking value for these user blogrolls, there is none, because google isn't allowed to visit those pages.
[ed. note: emphasis, and haughty 'we-told-you-so' tone, ours]

it is quite possible that the way those blogrolls is set up wasn't intended to deprive those blogrolls of any value.
[ed. note: yah, right. and duncan black doesn't mean to be snarky.]

i would disallowed the individual user pages if i had set up this site because the main page of those have the same content as the subdomain main pages for each user.

i also noticed that the main dailykos page that, it has a url canonicalization issue. see matt cutts blog post - seo advice: url canonicalization at http://www.mattcutts.com/blog/seo-advice-url-canonicalization/

if you have the google toolbar installed on your browser, and the pagerank indicator activated, you can look at the main domain address for the site with and without the "www" and see a difference in the numbers that it shows. with the "www" it appears to have a toolbar pagerank of 8. without the "www" it appears to have a toolbar pagerank of 7. it's possible that google doesn't recognize that they are in fact the same page, and may be splitting pagerank between the two pages. matt explains in his post how to resolve that issue with a permanent (301) redirect. since it's likely that most people linking to the site use the "www," then it may be best to redirect from the non=www version to the www version.
we are in great debt to mr. slawski in helping us prove that kos is a jerk blogroll purges are not healthy for greater blogtopia, and yes, we coined that phrase.

sorry, booman.

Labels: , ,

posted by skippy at 8:36 PM |

19 Comments:

Did we actually need more evidence that Kos is a jerk?
commented by Blogger Veronica, 9:06 PM PDT  
Well, Skippy you will never ever ever be purged from my blogroll.

:-)

This was a very well done post though, I remember how kind you were to me in helping me when I first got started blogging, back when I didn't even know what a page rank was and now I try to do the same for other bloggers that are starting. To me and to you and to a few of us, that's the way it's supposed to be...
commented by Blogger Lisa Renee, 9:11 PM PDT  
Maybe we need to think about a new name for this thing. It's about more than blogroll amnesty, and people are still repeating that inane "it's just a blogroll" and refusing to grasp that it's bigger than that, no matter how many times we (more or less) patiently explain it.

The first thought that comes to my mind is "Wankergate". But maybe that's because I'm tired and cranky. Should get to sleep now.

Thanks for following up on this, btw.
commented by Blogger Renee in Ohio, 10:21 PM PDT  
Great work skippy. A charitable reading of your findings would be that Kos and Atrios are completely ignorant of Search Engine Optimization, which I happen to know a little something about. One of the points of my original piece on "Amnesty Day" was to point out this ignorance and how it affects other blogs negatively. Recently, Atrios acknowledged in a tersely worded email that he actually had read my piece but he apparently either didn't believe what I was saying or didn't care. I hope your work will help persuade them that I was right. Whether they care or not is another thing entirely.
commented by Blogger Jon Swift, 10:47 PM PDT  
and why is the goal search engine optimization? is that really what kos and other big bloggers are for? that's not what i thought of them as.

my hits are way down, even though i made it onto the new atrios blogroll, and you know what? i really could care less.

but thanks for the link in your recent post/diary. i figure i gotta chime in here since you dubbed me the official blogroll amnesty day doubting thomas(tm)
commented by Blogger upyernoz, 6:19 AM PDT  
and why is the goal search engine optimization?
Prior to the purge, I would get hits from searches for people looking for opinions about Bush. Now those searches will mostly hit LGF and Freep networked blogs.

Recall, I wasn't on the Eschaton blogroll but I was on the blogroll of several people who were. My pagerank and others in liberal blogtopia have dropped. People are now more likely to find opinion pieces by reich wing blogs because of their SEO from freep and lgf.
commented by Anonymous George, 8:02 AM PDT  
i stayed on the atrios blogroll and my hits have still dropped. i was talking to the tech guy behind susie madrak's (suburban guerilla) site a couple of weeks back and he mentioned that blog hits are down overall for most sites, on both the right and left. the drop started around mid-2006. (see, for example, atrios' hits since may 2006)

the drop has more to do with aggregators and feeds than links. these days a person's hit count doesn't necessarily reflect the actual number of people who are reading

Prior to the purge, I would get hits from searches for people looking for opinions about Bush. Now those searches will mostly hit LGF and Freep networked blogs.

not true. try googling opinions about bush to see for yourself. no LGF or freepers are there at all. in fact, it seems to me that most of the first page search results are anti-bush, that's what hit #1 is. hit #3 is a raw story link. there's only a single bush supporting link on the first page, and it's at the very bottom.
commented by Blogger upyernoz, 10:56 AM PDT  
upyernoz, try google news, which is what we are talking about, not just google search.

google news search for opinions about bush include a heave emphasis from "free market news network" as well as arizona daily star (pretty conservative).

my mistake for not making that more clear.

also, i am sure george was not talking about the literal search "opinions about bush."

try news googling marcotte edwards. you'll get mostly right wing news organs, including world net daily, newsbusters, and the national cathoic register.
commented by Blogger skippy, 12:07 PM PDT  
oh it is true that rightwingers dominate google news. but i was under the impression that several prominent wingers are obsessed with getting their writings indexed there and are going out of their way to make it happen. i thought that was more funny and/or pathetic than something i would want "our side" to emulate. (see duncan's take here)

besides, i thought google news didn't index kos either.
commented by Blogger upyernoz, 12:16 PM PDT  
I have no idea whether or not this is something that can be looked into, but when you say blog-hits are down generally, I just assume it's because of the Dem takeover--I still read you guys a lot, but now I see lefty blogs as websites instead of as life preservers.

I would expect readership to have peaked in 2005, back when so many of us read blogs obsessively to convince ourselves that we weren't the only ones who saw how insane America had become.

None of which is to say that Kos and Atrios shouldn't eat shit.
commented by Anonymous the dryyyyyyy cracker, 12:51 PM PDT  
"i thought that was more funny and/or pathetic than something i would want "our side" to emulate."

Spoken like a true Democrat.
commented by Blogger Jon Swift, 12:56 PM PDT  
your point, mr. swift?

if you're trying to compare me with the ineffective dems in the national party, i reject the comparison. blogging is not going to change the world. for all but a select few it is and will always be a hobby. that's it. if we're lucky sometimes something we write might make some larger difference in the world but most of the time it won't. that's quite different from the power that a member of congress actually has.

for someone in congress to shrug their shoulders and say "well, there's nothing we can do" is wussiness, pure and simple. for me to do the same when talking about the effect of my blog is realism. believe it or not, a lefty blogger is not like a member of congress.

see the difference?
commented by Blogger upyernoz, 1:11 PM PDT  
I find it so amusing how often liberal bloggers sound like the cliched version of old-line conservatives. Too snobby and elitist to get down in the mud and dirty their hands. Condescending to the small and powerless. Dismissive of the grassroots and the power of individuals to drive the discourse. Overly obsequious to institutions and the rich and powerful. Yes, you would have made a fine conservative back in the day, Mr. Upyernoz. Nowadays, that kind of conservative is a rare beast. But it's a pretty apt description of more than a few liberals.
commented by Blogger Jon Swift, 2:12 PM PDT  
saying my blog doesn't have much effect on anything is snobby and elitist? how does that work?

i thought i had to be an elite to be an elitist. i'm not dismissing of the grassroots, i just think "grassroots" includes a lot more than just bloggers. and i'm not "obsequious to institutions and the rich and powerful" when i say that members of congress have political power that the rest of us lack. members of congress do have political power that the rest of us lack. that's why taking over congress was so important. if bloggers were more powerful than congress, we would have spent last fall working to take over the blogosphere.

and i find it amusing that someone who's schick is mimicking cliched versions of old line conservatives accuses me of sounding like a cliched version of old line conservatives.

seriously, we're having a disagreement here. it's nothing personal. i don't think i'm any better than you or skippy, we just happen to disagree.
commented by Blogger upyernoz, 2:33 PM PDT  
I think you misunderstood me. I was complimenting you, of course, by comparing you to conservatives. And I agree, it's nothing personal and I applaud you for your restraint in accusing me of having a schtick but not locating it in any part of my anatomy.
commented by Blogger Jon Swift, 2:44 PM PDT  
My traffic is actually up -- by quite a bit.

Doesn't mean all my previous criticisms of this ludicrous behaviour on the part of the Big Boys of Blogging are moot; to the contrary, it simply means that I am not beholden to them, lucky me, but the smaller blogs still got fucked when they did their despicable blogroll purge.

I doubt the fact that I have NOT been personally affected by this will give me any more credibility in the minds of the BBB or the BBB's Credulous Clamouring Crowds, though; to the contrary -- now, they'll just say I ought to shut the fuck up because I WASN'T affected, whereas before they said I ought to shut up because I WAS affected and therefore was only whining selfishly.

Some people you can't ever reach.
commented by Blogger Maryscott OConnor, 7:21 AM PDT  
MSOC,

i'm not saying shut the fuck up at all. i just disagree with you. whining about hits or perceived slights is not a very good way of convincing me, or anyone else who may disagree with you

jon,

d'oh. okay, sometimes i can miss the point. such are the hazards of arguing with a conservative such as yourself.
commented by Blogger upyernoz, 7:50 AM PDT  
Work at home opportunites For people who are smart enough to go after there own dreams

Work from home opportunites

Women who have children will love it...
commented by Anonymous Anonymous, 9:24 PM PDT  
This is a great Blog! But internet marketing costs money.
If you want to start for the price of a burger to supplement
your income you need a simple method. No PPC cost, no list!
Just some of your time! You can work at home with a system
that is as good as owning your own ATM Cash Machine!
ATM CASH
commented by Anonymous Anonymous, 9:54 PM PDT  

Add a comment