C# Web Bot

A bot will navigate websites and collect data and images in a similar way to bots already working for large companies such as Google, Yahoo! and others. The bot will be written in C#, using for example WebRequest,? and run from a desktop using an Internet connection. It will have the ability to navigate Javascript menus. One designation for a website is [[login to view URL]][1] Coders only need to test the bot and no actual production work will be done via the coders. ## Deliverables SPEC CHANGE: Due to a general response of people finding the business logic of the site very hard and that causing problems, I will divide the task. Therefore, this BID REQUEST only refers to grabbing the data without any understanding of the business data, but identifying the type of data from the website (eg a heading, a list element, plain text). The bot will navigate the Product Catalogue menus and submenus and then navigate to the page. It will collect all the data including the links and allow that data to be accessed in the C# program. That access will be handled by a new task, but the returned data must be in an understandable format, eg links, headings, list elements shown as they appear on the site. eg H1 Heading: Create Drift List Elements: Domestic Use, Modern Bathroom (eg <li> tag) Plain Text: This part is used for .... (eg <p> tag) Image Data: (img tag) Links: Href="....." (from link tags, including JS links) The site uses a lot of CSS so hard coded HTML tags do not exist.