if you were skeptical about the importance of a proper sitemap read this

[soaringeagle]

over the past couple months ive had a hard time keeping my sitemap updated with random issues interfering in the update process.

then superstorm sandy knocked out my power and messed up my pc (i had managed to get 1 update almost complete just before the power went out that was the only update in several weeks at least)

after that  my ram use climbed to 995 and stayed there so i upgraded to 32 gigs and it still  used all my ram  so the next few weeks i had tried to do updates i got 1/2 way through and had to reboot.

so finaly i reinstalled windows and everything today and am finaly on the way to a good update

so heres what ive noticed since not being able to update as often as i should

 

googlebots activity on my sites down by 505

black arrows are peaks after the last 2 complete updates there was no peak after the incomplete update when sandy hit most likely because there were fewer urls in the sitemap then the previos update (bout 200k less)

red arrow is the average activity level when i am able to update regularly

 

now how its affecting trafic;

my trafics been on a steady rise slow but sure

well its just taken a huge dip 1000 less visits a day just the past few days

dropping below levels it hasnt been below in the past 6 months

(no graph included just cause its hard to see the trend unless you know what to look for)

 

ps the sitemap generator i recomend is Sitemap Creator 4if you have a windows machine to run it on

also interesting point  during  the same time period the number of number 1 ranking pages and pages on page 1 of google results ..varied but the averages didnt change enough to explain the drop

however  the number of pages indexed  did go down by a big factor (200k) and the freshness of pages was reduced (meaning pages showing up in top 5 positions within an hour or 2 of posting)

 

i cant stress enough the importance of an updated fresh  and proper sitemap

its the cheapest easiest seo you can do

it wont replace on or offsite seo work but should supliment it and  will give ya the most bang for the buck

since its a 1 time charge to get the software then u just run it over and over and watch the trafic increase

you can spend hu8ndreds a month  on seo  doing page optimization and link building

but see very lil and extremely slow results (especialy with link building)
but having a freshly updated sitemap  and having it ping the search engines when its updated will get those bots to come running to see whats new  and get it indexed faster  and more dependably.

 

most likely you will see a 505 increase within weeks  and a steady increase from there  as well as faster indexing (alreasdy very busy sites might not see 505 but should still see a noticable increase)

 

 

just a few more words about sitemaps;

• ning sites are huge  a brand new site with almost no content would be around 2k urls the ning sitemap for that site might list 3 or 4..maybe 5 the max any ning sitemap would list is around 10 urls...only the features (forums blogs photos) nothing within the features a "proper sitemap" listes every single page you want crawled (every page not blocked by robots.txt)
• creators is huge 24,000 members and so many discussions added a day more then 1/2 never get a reply google lists 300k pages of creators my site is under 10k members  its active  and big but nowhere near as active and fast moving as creators.. google indexed 3 million pages of my site
• imagine going to a city youve never been to, a million or more criss crossing streets  zig zagging  every wich way now imaging this entire city with  5 million streets 100 million addresses and only 4 road signs  can you find the coffeehouse you heard about or  the musuaem or hell can you even find your way home or out of the city for that matter? a sitemap is like a map of thesite or city it tells the bots where to go for the best treats where to avoid  unless looking for a seedy dark back alley, where te attractions are and whats not at all important to see

since upgrading to Sitemap Creator 4with the new video sitemaps ive only run it once and that was the incomplete run but in the past couple weeks ive noticed twice as many vids indexed and several dozen more indexed daily (i have vids set to a semi low priority and change frequency since its a way lower activity on y site ..50k photos and aboput only 2k videos so set pics to 0.9  change freq always and vids to 0.3 and weekly

 

note the vids were indexed as pages already but not as vids

so including all content types in the sitemap not only every page but  every file embeded in every page etc makes a big diference

 

now that my pc is runing better again ill be updating my sitemap weekly (takes about 5 days each run) once i have several done fairly steadily instead of just now and then my trafic should bounce right back and increase again  tye google bot activity should increase as well

and once again should see pages posted show up at the top of searches within minutes to hours instead of days to weeeks

(when the site was smaller and it only took 24 hours to run an update i had seen pages show up in top 3 positions within 3-5 minutes even)

 

pss (i know i rambled alot) see my older sitemap tips and my google domination tips for how to do a proper sitemap on ning since nit only does nings sitemap fall very short from being a proper sitemap, but ning also makes it harder to do one properly..in fact Sitemap Creator 4 has a url replacement feature (the only sitemap creator i know of that has it) that i had them add to it just to make it work with ning (no root access means you have to use a subdomain or other domain and use url replacement to 'fix" your sitemap indexes)

 

[NING left by lousy service]

How do I send to the root folder of my sitemap?

[soaringeagle]

you cant but  if you see my other sitemap tips the sitemap crator 4 allows you to put sitemaps on a subdomain and use url replacement to fix the indexes

[NING left by lousy service]

I downloaded this trial program, how do I export the sitemap? Or I have to buy the program to do this?

[soaringeagle]

you can with trial but are limmited to  only like 10k urls or maybe only 1k i forget

u need a subdomain or other domain to host the sitemaps though

[NING left by lousy service]

I just received a ticket holder Ning informing me that I edit my sitemap at this link:

http://yournetwork.ning.com/main/admin/sitemap

Editing is possible here?

[soaringeagle]

only foor very small sites nings siytemap is useless

u can paste is a sitemap there if its under 40k urls after that u must use a sitemap index and multiple siyemaps thats why ypu shoud set it up now on a subdomain

[NING left by lousy service]

I have subdomain that I can use, also did a search with site Crator 4 and he captured over 400,000 pages and had to shut it, but there is still all urls. How do I export? What should I post on subdomain?

thank you very much

[soaringeagle]

1st step import the robots txt intoi the excluded pages so you wont crawl pages tyou dont want

next crawl if it goes past 400k let it go past 400k let it finnish

nextin the upload settings set the ftp of the subdomain (the subdomain must be verified in webmaster tools ) (ps put api.ning.com as a alternate root in u-images  select no to[itle no descrio\ption)

on upload settings  put the subdomain in url replacement

once the scans done  click upload  or create sitemap and upload by ftp there select all the options  u want an d set split  sitemaps at 40k not 50k urls enable sitemap index (gz outputs good too)

it wikl upload to the subdomain

now in your robots.txt put this lik=ne 1st

Sitemap; http;yoursubdomain.sitename.com/sitemap.gz (url to your sitemap)

[soaringeagle]

read over all muy other sitemap discussions for more details

[NING left by lousy service]

Thank you very much!!!