over the past couple months ive had a hard time keeping my sitemap updated with random issues interfering in the update process.
then superstorm sandy knocked out my power and messed up my pc (i had managed to get 1 update almost complete just before the power went out that was the only update in several weeks at least)
after that my ram use climbed to 995 and stayed there so i upgraded to 32 gigs and it still used all my ram so the next few weeks i had tried to do updates i got 1/2 way through and had to reboot.
so finaly i reinstalled windows and everything today and am finaly on the way to a good update
so heres what ive noticed since not being able to update as often as i should
googlebots activity on my sites down by 505
black arrows are peaks after the last 2 complete updates there was no peak after the incomplete update when sandy hit most likely because there were fewer urls in the sitemap then the previos update (bout 200k less)
red arrow is the average activity level when i am able to update regularly
now how its affecting trafic;
my trafics been on a steady rise slow but sure
well its just taken a huge dip 1000 less visits a day just the past few days
dropping below levels it hasnt been below in the past 6 months
(no graph included just cause its hard to see the trend unless you know what to look for)
ps the sitemap generator i recomend is Sitemap Creator 4if you have a windows machine to run it on
also interesting point during the same time period the number of number 1 ranking pages and pages on page 1 of google results ..varied but the averages didnt change enough to explain the drop
however the number of pages indexed did go down by a big factor (200k) and the freshness of pages was reduced (meaning pages showing up in top 5 positions within an hour or 2 of posting)
i cant stress enough the importance of an updated fresh and proper sitemap
its the cheapest easiest seo you can do
it wont replace on or offsite seo work but should supliment it and will give ya the most bang for the buck
since its a 1 time charge to get the software then u just run it over and over and watch the trafic increase
you can spend hu8ndreds a month on seo doing page optimization and link building
but see very lil and extremely slow results (especialy with link building)
but having a freshly updated sitemap and having it ping the search engines when its updated will get those bots to come running to see whats new and get it indexed faster and more dependably.
most likely you will see a 505 increase within weeks and a steady increase from there as well as faster indexing (alreasdy very busy sites might not see 505 but should still see a noticable increase)
just a few more words about sitemaps;
since upgrading to Sitemap Creator 4with the new video sitemaps ive only run it once and that was the incomplete run but in the past couple weeks ive noticed twice as many vids indexed and several dozen more indexed daily (i have vids set to a semi low priority and change frequency since its a way lower activity on y site ..50k photos and aboput only 2k videos so set pics to 0.9 change freq always and vids to 0.3 and weekly
note the vids were indexed as pages already but not as vids
so including all content types in the sitemap not only every page but every file embeded in every page etc makes a big diference
now that my pc is runing better again ill be updating my sitemap weekly (takes about 5 days each run) once i have several done fairly steadily instead of just now and then my trafic should bounce right back and increase again tye google bot activity should increase as well
and once again should see pages posted show up at the top of searches within minutes to hours instead of days to weeeks
(when the site was smaller and it only took 24 hours to run an update i had seen pages show up in top 3 positions within 3-5 minutes even)
pss (i know i rambled alot) see my older sitemap tips and my google domination tips for how to do a proper sitemap on ning since nit only does nings sitemap fall very short from being a proper sitemap, but ning also makes it harder to do one properly..in fact Sitemap Creator 4 has a url replacement feature (the only sitemap creator i know of that has it) that i had them add to it just to make it work with ning (no root access means you have to use a subdomain or other domain and use url replacement to 'fix" your sitemap indexes)
I agree, the traffic is not great. Something wrong with the schematic infrastructure of the network design. For starters, the forwarding of the domain from mysite.ning.com to mysite.com is a problem with Google webmaster, and secondly, having so many similar network on the same server, or root domain can't be good.
If one site got banned from Google, it probably affected the other networks, specially if that network is a porn network.
Prior to the sitemap you introduced, Google would only indexed 80 pages, and everyday I have to sort out errors. While your sitemap indexed 80,000 pages, they are not ranking high, they seems to be at the bottom of the search page.
Most of our pages are SEO...Now. My Google Webmaster showing over 540 server error message, and 8,777 not found pages. All these server error, and page not found can't be good for the network.
I think the only way our network will progress is to be on a open source scrip which we working on.
dont worry aboit those thopse are normal when the server goes down or into maintenance mode or discussions are deleted or people leave or get baned your gouing to have those
no theres no issues with the forwarding or the domains on the same ip
yes havingyour own ip gives u a tiny tiny advantage but its so lil to not be worth mentioning
if you say the pages are seo maybe the seo youve doe isnt working or is working against you?
google bans by domain
ip only i believe if theres malware or spamming widespread by that ip
How do you indicate into webmaster tools the sitemap link???
I got "www.domain.com/"... already written...
Thanks a lot
24 hours for 300.000 pages... it's gonna take a week to crawl... but I'm excited to test the effect of this sitemap ;-)
thats actualy really fast are u getting timeouts? i crawl at 200k pages in 24 hours and thats bout fastest i can go with accurate results
No timeout... as I''m writting : 32:40:00 -> 411700...
ok lil fast but not too bad how many threads and what crawler delay
I've set everything like you show on your post image sitemap...