Advertisement
If you have a new account but are having problems posting or verifying your account, please email us on hello@boards.ie for help. Thanks :)
Hello all! Please ensure that you are posting a new thread or question in the appropriate forum. The Feedback forum is overwhelmed with questions that are having to be moved elsewhere. If you need help to verify your account contact hello@boards.ie

Website crawl

  • 10-10-2012 1:31pm
    #1
    ilds
    Registered Users Posts: 3 ✭


    Looking for someone to crawl two websites, PM me for more details...


Comments

  • #2
    tricky D
    Closed Accounts Posts: 9,700 ✭✭✭tricky D


    What more precisely do you mean by crawl?

    And how many pages in each site?


  • #3
    TsuDhoNimh
    Registered Users, Registered Users 2 Posts: 424 ✭✭TsuDhoNimh


    Going one further than Tricky asking what you mean by crawl, I'd take it back a level and ask what are you hoping to achieve?

    If it's something 'easy' that someone will be able to provide with the limited information given to date, it's more than likely something that can be done free of charge and allow you the freedom to do it yourself at any time in the future should you wish to.

    If it's something more complicated...
    ilds wrote: »
    ... PM me for more details...

    Perhaps you'll get lots of PM's offering help here, but I'd doubt it. Without more information on what you're actually looking to do I very much doubt anyone will take the time to PM you to figure it out (I questioned taking the time to post to do it).

    I use crawlers that range from home spun, free to use (private or commercial) right up to expensive tool kits and I'm still not sure if I'd have anything that meets your needs. So more information is definitely required. There are a huge number of reasons that you might wish to crawl a site, ranging from completely legitimate and above board right down to highly illegal, so without more information the feedback you get will be extremely limited.


  • #4
    off.the.walls
    Registered Users, Registered Users 2 Posts: 1,298 ✭✭✭off.the.walls


    Crawl??


  • #5
    smash
    Closed Accounts Posts: 34,809 ✭✭✭✭smash


    off.the.walls wrote: »
    Crawl??
    scrape content... pretty dodgy practice!


  • #6
    tricky D
    Closed Accounts Posts: 9,700 ✭✭✭tricky D


    might also be index (the old fashioned way) or spider view or.................


  • Advertisement
  • #7
    ilds
    Registered Users Posts: 3 ✭ilds


    I think scrape content is probably the most accurate description, there could be up to around 100 pages.....


  • #8
    mneylon
    Registered Users, Registered Users 2 Posts: 7,740 ✭✭✭mneylon


    ilds wrote: »
    I think scrape content is probably the most accurate description, there could be up to around 100 pages.....

    If it's not your content then you're basically asking someone to potentially break the law ..

    Blog | Tweets



  • #9
    CBIL360
    Registered Users Posts: 5 ✭CBIL360


    What exactly you want?
    Looking for someone to crawl two websites
    It is usually done by search engines, Do you want some one who can go through and analyze your site?


Advertisement