The next logical step after getting wordpress mu to work with multiple domains is to tell google all about your sites via google compliant sitemap.xml files. If you’re anything like me, you would have searched through the mu forums and come out more confused than what you started with. There’s about three attempts to make a mu plugin over at wpmudev.org and they all fail dismally. This is so not typical of wordpress as normally, there’s a plugin developed for just about anything you can think of. With such a gap in the mu works, I’ve had to hack my own sitemap plugin to get my domain parking mu working. Luckily, Arne Brachhold’s plugin for the normal wordpress is very nicely coded and has all the bells and whistles that you’d ever what a sitemap plugin to have.
The first problem with Arne’s plugin is figuring out where to put the sitemap.xml since you have so many blogs running off the one wordpress. If you put it in your root directory, it will be overwritten by every blog as they are all pointing to the same location. The first hack is to default the location of the output to each blog’s individual upload folder (ie. /wp-content/blogs.dir/
Edit the InitOptions() function in the sitemap.php file
//JH brute force - set the sitemap location to manual mode
$this->_options["sm_b_location_mode"]="manual"; //Mode of location, auto or manual
//workout what the upload directory for this blog is
$upload_dir = ABSPATH . UPLOADS;
//create the directory if it does not already exist
wp_mkdir_p($upload_dir);
//preset the manual directory and url
$this->_options["sm_b_filename_manual"]= $upload_dir . "sitemap.xml";
$this->_options["sm_b_fileurl_manual"]= trailingslashit(get_bloginfo('siteurl')) . "sitemap.xml";
Next we want to force it to output a robots.txt file every time so that the search engine spiders know where to find the sitemap. To do this we set the robots flag to true in the same initOptions function.
$this->_options["sm_b_robots"] = true;
There is also a bug in the GetRobotsFilePath() function in that it is not putting it in the same location as the sitemap path. Another dirty hack is needed:
function GetRobotsFilePath() {
//return trailingslashit($this->GetHomePath()) . 'robots.txt';
//JH brute force
return ABSPATH . UPLOADS . "robots.txt";
}
That’s it for the sitemap.php file.
The next hack is to wordpress itself. Wordpress mu is only up to v2.3.1 of wordpress and there is a bug in one of the functions to write to a file. Whilst we are waiting for mu to update, this silly line needs to be commented out from one of the functions.
function insert_with_markers( $filename, $marker, $insertion ) {
//return;
Yep, that’s exactly right. Someone’s put a return statement as the first line of code so that this function exits straight away without actually creating and editing the file. This function is used to create your .htaccess file and the sitemap plugin reuses it to write your robots.txt
We’re almost there. Google doesn’t like your sitemap.xml file to be anywhere except your root directory. It is currently sitting in http://yourblog.com/files/sitemap.xml and we want to actually trick google to see it as http://www.yourblog.com/sitemap.xml. To do this, we need to edit the .htaccess file on your root directory:
#uploaded files
RewriteRule ^(.*/)?files/$ index.php [L]
RewriteRule ^(.*/)?files/(.*) wp-content/blogs.php?file=$2 [L]
#sitemap
RewriteRule ^sitemap.xml$ wp-content/blogs.php?file=sitemap.xml [L]
RewriteRule ^sitemap.xml.gz$ wp-content/blogs.php?file=sitemap.xml.gz [L]
RewriteRule ^robots.txt$ wp-content/blogs.php?file=robots.txt [L]
Those three lines under #sitemap will redirect calls to the files to your blogs upload directory. The blogs.php file is mu’s handler to redirect to the right blog directory depending on what domain you are accessing.
Phew that’s it! Hope I didn’t lose anyone. It could have been a bit cleaner but it’s a hack specifically for my wordpress mu so it doesn’t need to be pretty :) All that there is to do now is to login to your wordpress, enable plugins for your blog, activate the sitemap plugin and generate your first sitemap.
O. And when you submit your sitemap to google’s webmaster tools, you should use the meta tag option to verify your site rather than the file option. This avoids having hundreds of google2143293udsf0.html files in your root directory. An easy way to do this is the Add Meta Tag plugin. Remember, both the add Meta Tag plugin and the Sitemap plugin go in the wp-content/plugins directory as they are just normal wordpress plugins.
Update 26/6/2008: I’ve finally gotten some free time to clean up this hack and upload it so that you don’t have to edit the code anymore. Download it here on my new bitminds software site.
Popularity: 20%
Tags: hacks, sitemap, wordpress mu
30 comments ↓
GREAT STUFF! You are absolutely right, it is a bear trying to find out a workable solution for mu sitemaps… my next read is your multi site manager…. that is HUGE for me…..if I can get it working.
Hi buddy,
Great work!
I almost did it by following your instructions. Took long enough time for this newbie to see some rays of hope. Thank you so much. :)
Just one question:
Which folder/file in WPMU has to be modified to get this corrected
function insert_with_markers( $filename, $marker, $insertion ) {
//return;
I couldn’t find this function in the wp-includes/functions.php file.
Hi Sam!
It’s in admin/includes/misc.php but it’s fixed in mu v1.3.3 which just came out a few days ago.
Hi Jerry,
Thanks for the helpful reply. I know you are not here to solve everyone’s problems. Just in case you don’t mind.. I am facing a problem. When I access the sitemap using Internet explorer, I am able to see a well formated sitemap. But when I try to access it using Mozilla, it asks me to download it. I am not an expert, so I have no clue about why this is happening. I was using Arne’s sitemap generator in a single user wordpress install, and I was able to view the sitemap even in mozilla. Don’t know why I am not able to do that in wpmu. Could you help please..
In case you want to have a look at the problem, here are some of my wpmu installs sitemaps:
http://thinkmate.in/b/sitemap.xml
http://anuroop.thinkmate.in/b/sitemap.xml
The above sitemaps are generated using your hacks. They really don’t exist in the /b/ folder.
Hi Sam,
your sitemaps look okay to me. It might just be how IE and Firefox handles xml’s. IE likes to display them and Firefox doesn’t. What’s more important though is whether google likes them or not. Go verify your sitemap on http://google.com/analytics/ and see if it passes.
After instaling XML Sitemap Generator for WordPress 3.0.3 on WPMU.1.3.3 we are receiving the following problems:
1. As we generates the sitemap at the plugin page for the first time on each created blog, after indicating the sitemap creation the blog change to the principal one.
2. As we safe the options at the plugin page we receive the message at the top:
Error while saving Robots.txt file
3. As we register at Google webmaster:
a) http://my.domain.com/sitemap.xml (principal)
b) http://my.domain.com/other/sitemap.xml (others)
We receive at Google webmaster site the message:
Unsupported file format. Your Sitemap does not appear to be in a supported format. Please ensure it meets our Sitemap guidelines and resubmit.
We are newbies and will appreciate your help.
Best regards,
Hi ‘Admin’, unfortunately I don’t have a mu installation using subdirectories. I’d imagine the above instructions in this post to also work except for the htaccess bit where you will need to tweak it a bit.
If there’s a lot of requests, I might consider working at it and rolling all this up into its own package instead of having everything hack away at the code like I did.
I’ve also helped a couple of people out individually but I suspect that my charge out rates may be a bit prohibitive to you if you are just a newbie starting out in the big bad o’ jungle of making money online ;)
Best of luck though. Jerry
This worked like a charm! Thanks very much….
It worked nicely, thank you! ;)
One question: if I type the sitemap url in the address bar, the sitemap will be downloaded, instead of being shown in the browser… is this a problem?
Hi Notoriousxl.
As long as the contents of the xml can be verified by google, it doesn’t matter whether your internet browser shows it on a page or asks you to download the file.
Jerry, Thank you very much for this article. This is exactly what I was looking for. It worked like a charm!
It didn’t work for me using mu 1.5 (RC1) and xml sitemaps 3.0.3.1. The plugin actually worked for the main blog until I made the changes. It wouldn’t update settings on subblogs (just revert to the main blog), and the changes caused the MU blog sign up page to load with or without the htaccess modified. Once I made the changes to the sitemap.php file, the main blog site map was also broken.
[…] for WordPress. The problem is it doesn’t support WordpressMu out of the box. So I found this article posted by Jerry Huang that explained how to make this plug in WordpressMu compatible. After a […]
So to be clear - as I have yet to get this to work in “mu”:
But want it to - oh so badly !!!
You say:
Edit the InitOptions() function in the sitemap.php file
//JH brute force - set the sitemap location to manual mode
$this->_options[”sm_b_location_mode”]=”manual”; //Mode of location, auto or manual
//workout what the upload directory for this blog is
$upload_dir = ABSPATH . UPLOADS;
//create the directory if it does not already exist
wp_mkdir_p($upload_dir);
//preset the manual directory and url
$this->_options[”sm_b_filename_manual”]= $upload_dir . “sitemap.xml”;
$this->_options[”sm_b_fileurl_manual”]= trailingslashit(get_bloginfo(’siteurl’)) . “sitemap.xml”;
That is as in drop all of this “untouched” into Line# 1296 ?
Hi,
Thanks for helping all :)
I am getting this error:
“There was a problem writing your sitemap file. Make sure the file exists and is writable”
what I am missing?
Liron
@Mu User, you have to replace the bits of lines that initiate the values for sm_b_location_mode, sm_b_filename_manual, and sm_b_fileurl_manual.
@Liron, check that your upload directory is writable. If you change the permissions to 777 and it is still not working then I would suspect that you’ve make a mistake somewhere in the code. I haven’t had to change my directory permissions though and it still works.
Has anyone tried this with WPMU 1.5.1?
Do the instructions change at all?
[…] I also chosen to place the file in a separate domain (BitMinds.com) so that it is easier to find. I plan to put my other source codes here when I have time to clean them up (eg. my photo battle script), and maybe my own customised wpmu google sitemaps generator… […]
Hello, I have installed it following your instructions and, it seems to work correctly under the blogs hosted under xxxx.acuablogs.com, but when I try to access to root blog sitemap.xml I get a 404, do you know were the problem can be?
look:
working: http://kizd.acuablogs.com/sitemap.xml
not working: http://acuablogs.com/sitemap.xml
Thanks.
I’ve keep checking back to see if you answered comment # 17:
“Has anyone tried this with WPMU 1.5.1?”
“Do the instructions change at all?”
Looking forward to using the sitemap plugin.
Thanks in advance.
@keith, I’ve upgraded to wpmu 1.5.1 now and it works fine.
@stephen, have you checked to see if the file got created in blogs.dir/1/files/ ? If so then it may be your htaccess issue. Hard to tell without going through your setup though
hey Jerry, in blogs.dir/1/files is not created but in blogs.dir/7/files that it s the another blog I put in previous post it is created. I ll check for permissions.
If you think sth can help, it will be apprecciated.
Thank you very much.
Google SiteMap For WordPress MU Plugin 1.513101
http://www.richardpalace.com/2008/06/25/google-sitemap-for-wordpress-mu-plugin-1513101/
now available for download here: wpmu google sitemaps generator
I have tested this theory time and time again, and it may or may not help make this process MUCH easier.
But… Google for example doesn’t care what the file extension is of your sitemap. It doesn’t have to be .xml, it can be .php or .asp for example.
All that the search engines actually care about is the fact that the file output is in the correct format and that the file is placed in the root folder. So, in the robots.txt file and google webmaster tools, all you have to do is give the url of your dynamic sitemap file.
Going through the help sections of both google and sitemaps.org, its examples show the xml extension but never states that the file has to end with xml.
So, in theory this process can be simplified if you are able to place a sitemap.php file in the root folder. Then within the sitemap.php have it pull back the posts/pages based on the domain name and what is in the db.
The process of having the sitemap automatically submitted to google doesn’t even have to change. Other then making sure to submit the correct file name.
Nice this worked Great!
Thanks every thing worked fine key is to just take your time and think it thru
The web is great place made by people like you
Bill007
Thanks Jerry for the updates of the plugin. I have one Question: The plugin works fine with WPMU and when using the PluginCommander The plugin even can be activated for the users. My question is How can I generate the XML with the activation of the plugin, so the users will not have to get into it and create the sitemap for each blog?
Updates robots.txt, but doesn’t actually generate the XML files.
This message appears:
The sitemap wasn’t built yet. Click here to build it the first time.
But when I click on it, it doesn’t do anything. :(
d00d, totally sweet, many thanks and appreciation for taking the time to tweak arne’s pluging and making it work! Saved much much time, thanks again!
I have installed the plugin and activated it, but when I run it the xml files and robots.txt files are created, however the plugin still says: “The sitemap wasn’t built yet. Click here to build it the first time.”
Can anyone help me get this plugin working with WP MU version 2.6.1. I have the wpmu plugin version 3.0.3.3(modified for wpmu).
Thanks.
Leave a Comment