![]() |
|
![]() |
#1 |
Green Mole
Join Date: Jan 2007
Posts: 1
|
Index some, but spider all pages
Hi all,
Have a Question: The index page of the domain i want to spider, contains links to say level1.php?id=xx and level1.php?id=yy level1.php?id=xx contains links to level2.php?id=aa and level2.php?id=bb level1.php?id=yy contains links to level2.php?id=cc and level2.php?id=dd The pages i want to index are the level2.php pages. But i only can reach them trough index -> level1.php -> level2.php. So the spider needs to follow all the links from the root. What would be the best way to index some, but spider all pages? |
![]() |
![]() |
![]() |
|
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
Spider stops before all pages are indexed | halide | Troubleshooting | 3 | 07-19-2005 12:26 AM |
Using a dictionnary to spider pages | Edomondo | How-to Forum | 0 | 11-23-2004 07:36 AM |
converted from html pages to php pages now no pages will index!!! help!! | bigals | Troubleshooting | 24 | 04-01-2004 09:34 AM |
Remove Index pages | bigals | How-to Forum | 11 | 03-12-2004 12:16 AM |
do not index all pages | robilix | Troubleshooting | 2 | 11-25-2003 01:50 PM |