ASP question

Panda

lo there,
im in the thick of an asp project and i'm a bit stuck.

my project is partially based around scheduling, mainly television.
i want to retrieve the television listings from aertel whenever they're updated,
and then display the listings on one of my own pages.

is this possible?
how would i go about retrieving the data?

damnyanks

You can use RSS feeds.

That would be the easiest way...

Just a random guess type thing but you could program a robot (Like google bot and spam bots so on) To crawl through a tv listings site and parse it all somehow into a nice lil ol output screen


But the RSS feed would be alot easier

The Corinthian

As has been suggested, RSS feeds or similar would be the easiest. Call them up and tell them it’s for a project and they may give you access to such a feed.

The alternative is a dirty hack that is a favourite of those of us who have had to work with legacy systems known as screen scraping. Essentially this involves reading the output from another application and parsing it to pick out the necessary content.

As you’re using ASP, I would suggest you look at a component such as the XMLHTTP object to grab the content of a remote URL. A simple tutorial can be found at:

http://www.4guysfromrolla.com/webtech/110100-1.shtml

Enygma

I've been looking into something like this too. The Radio Times website is supposed to have a complete listing of TV programs but I can't find it in machine readable format.

Take a look at the XMLTV project on sourceforge.net

Panda

thanks a million lads, i've looked at the stuff ye have provided and it looks very promising.

just what i wanted.

thanks again.

Panda

/bump

As someone in the other asp thread noticed, projects are due up soon enough.

I couldnt get the code i found to work for aspTear, the code was referring to a file called System.dll created by and not supplied by the person who supplied the snippet of code.

bastid.

i've had to move on as i cant find any other good example of code.

my lecturer said i should try parsing the html page and place the data into an access database. Then use asp.net to display the database.

Can anyone tell me where to start? I searched for code all yesterday and i cant find a clear example of code.

the page i need to parse is isis.rte.ie/rteguide


help!

The Corinthian

Originally posted by Panda
As someone in the other asp thread noticed, projects are due up soon enough.

That was me. It’s easy to spot, you get a deluge of students on Usenet, mailing lists and Boards.ie looking for people to do their homework for them, given they’ve at best spent an hour on it themselves to date, gave up and then waited until the last minute before returning to it.

I couldnt get the code i found to work for aspTear, the code was referring to a file called System.dll created by and not supplied by the person who supplied the snippet of code.

ASPTear was originally written for use on NT4 and with ASP2.0. It’s quite likely that it’ll have numerous issues with newer platforms. Use the XMLHTTP as I suggested earlier in the thread, or even something more up to date than that.

my lecturer said i should try parsing the html page and place the data into an access database. Then use asp.net to display the database.

He means screen scraping. That was already been suggested to you weeks ago. You’ll still need a component to do the remote HTTP GET for you.

Can anyone tell me where to start? I searched for code all yesterday and i cant find a clear example of code.

An example or the code already written for you? Search www.4guysfromrolla.com for a relevant article.

the page i need to parse is isis.rte.ie/rteguide

Have you called them to ask if you could access an RSS feed from them for your project as was suggested here? That would make your life a lot simpler.

Panda

okay, thanks again.
i have the code running now.
the whole page is scraping just fine, now i just need to cut the crap out that i dont need.

much appreciated.

by the tone of your text, you must be kinda sick of people asking for help.

for the record this isnt a major part of my project at all, thats why its been put off.
the main part involves sms messaging and i have the main part already done.

/edited mistakes.

Panda

Sorry to bother you again.

The code that they provide works fine but it takes the whole page,

the page i'm scraping is http://www.rte.ie/aertel/p171.htm to start off with.
is there any way i can get rid of everything but the schedule?

thanks again.

The Corinthian

Originally posted by Panda
the page i'm scraping is http://www.rte.ie/aertel/p171.htm to start off with.
is there any way i can get rid of everything but the schedule?

Yes, it's called programming. It's like cut and paste, except you write it yourself.

Assuming you're using VBScript for your ASP scripts, parse the raw HTML using the InStr and Mid functions. Find some part of the code that allows you to identify the start and end of the content you want (hint: how do you identify it yourself?) and then extract it. Once you've done that clean it up by removing all the tags. Those to functions are actually all you need to do it.

Panda

<%@ Page Language="vb" Debug="true" %>
<%@ import Namespace="System.Net" %>
<script runat="server">

Sub Page_Load(sender as Object, e as EventArgs)
'Create a WebClient instance
Dim objWebClient as New WebClient()

'Call the DownloadedData method
Const strURL as String = "http://www.rte.ie/aertel/p171.htm&quot;
Dim aRequestedHTML() as Byte

aRequestedHTML = objWebClient.DownloadData(strURL)

'Convert the Byte array into a String
Dim objUTF8 as New UTF8Encoding()
Dim strRequestedHTML as String
strRequestedHTML = objUTF8.GetString(aRequestedHTML)

'display the string
lblHTMLOutput.Text = strRequestedHTML
End Sub
</script>
<html>
<head>
<title>Todays RTE1 Schedule</title>
<link href="rmstyle.css" type="text/css" rel="stylesheet" />
</head>
<body>
<div align="center">
<h3>Todays RTE1 Schedule
</h3>
<asp:label id="lblHTMLOutput" runat="server"></asp:label>
</div>
</body>
</html>


Still stuck, please help.
This part works fine its taking the information from the page but i dont know what to do with the parsing.
Can you be more exact as to where i need to use the mid and instr?
and how do i pass the values to the database?