[html ]Why can't I see the table data when I view source?

Brussels Sprout

I'm looking to write a simple application that will update my local fantasy football league scores. Presently we're using a spreadsheet which needs to be updated manually.

I'm looking to write a screen scraper app and I've found a website that has the information that I'm looking for however I can't seem to find the displayed data when I view the source html.

Here's the website: http://fpl.checkmystats.org/

I want to retrieve the Pts column in the displayed table for particular players. When I view the page source though the <tbody> section of the table is empty.

I presume this means that the data is being inserted by javascript but I can't figure out what the original source of the data is. Is there another url in there that I'm missing?

duffman85

$(document).ready(function() {

    var team_badge = {"Arsenal":1,"Aston Villa":2,"Cardiff City":3,"Chelsea":4,"Crystal Palace":5,
    "Everton":6,"Fulham":7,"Hull City":8,"Liverpool":9,"Man City":10,"Man Utd":11,"Newcastle":12,
    "Norwich":13,"Southampton":14,"Stoke City":15,"Sunderland":16,"Swansea":17,"Tottenham":18,
    "West Brom":19,"West Ham":20};

    oTable = $('#example').dataTable( {
        "bProcessing": true,
        "bServerSide":false,
        "sAjaxSource": [B]'data-json.php',[/B]
        "sDom": "<'rowf'<'span6'l><'span6'f>r>t<'row'<'span6'i><'span6'p>>",
        "sPaginationType": "bootstrap",

It's all being pulled in from data-json.php

Talisman

A quick look at the source code reveals that oTable loads the data via a call to another page. Have a look at line 230:

"sAjaxSource": 'data-json.php',

Graham

Yup, JSON date source:

e.g http://fpl.checkmystats.org/data-json.php?_=1384016717865

I think the number in the querystring is your session ID

Brussels Sprout

Super-thanks guys!

benny306

Hi,

Did you get a working script yet? I've been working on something similar and am using http://fantasy.premierleague.com/web/api/elements/ to scrape player stats.

You just need to put the player id at the end of the web address to get stats. For example http://fantasy.premierleague.com/web/api/elements/1/ will list all the stats for Lukasz Fabianski.

I'm just learning but have managed to get it working using the below (**I know my code probably isn't the greatest, but it seems to work). I still haven't set up a database to store the data yet

<?php
ini_set('max_execution_time', 300);
$id = 1;
for ($id=1; $id<=10; $id++)
{

$endhash ='/';
$url = 'http://fantasy.premierleague.com/web/api/elements/';
$urlid = $url . $id . $endhash;
$results = file_get_contents($urlid);
$playerstats = json_decode($results, true);
}
?>

Then if you want to pull player name and points I would put the below inside the for loop above:

echo $playerstats;
echo $playerstats;

moikey

benny306 wrote: »

Hi,

Did you get a working script yet? I've been working on something similar and am using http://fantasy.premierleague.com/web/api/elements/ to scrape player stats.

You just need to put the player id at the end of the web address to get stats. For example http://fantasy.premierleague.com/web/api/elements/1/ will list all the stats for Lukasz Fabianski.

I'm just learning but have managed to get it working using the below (**I know my code probably isn't the greatest, but it seems to work). I still haven't set up a database to store the data yet

<?php
ini_set('max_execution_time', 300);
$id = 1;
for ($id=1; $id<=10; $id++)
{

$endhash ='/';
$url = 'http://fantasy.premierleague.com/web/api/elements/';
$urlid = $url . $id . $endhash;
$results = file_get_contents($urlid);
$playerstats = json_decode($results, true);
}
?>

Then if you want to pull player name and points I would put the below inside the for loop above:

echo $playerstats;
echo $playerstats;

Hi benny306, do you know where I can get a list of player ids or API queries for the premier league fantasy game?

benny306

Hi Moikey,

I couldn't find any API queries in my search so I'm not sure if there are any.

For player ids, I have a list of 675 player ids in my database I could give you. Are you looking for specific players or just a list of the entire players database ids?

moikey

benny306 wrote: »

Hi Moikey,

I couldn't find any API queries in my search so I'm not sure if there are any.

For player ids, I have a list of 675 player ids in my database I could give you. Are you looking for specific players or just a list of the entire players database ids?

You could PM me them thanks:), is there any other ways of sorting the JSON say team specific or just points specific. I cant find any info from the premier league site. Is this an API of the online game or just player specific?

benny306

Did you have something specific in mind as to what you wanted to do with the API?

I've attached a pdf of the database so you can see player ids. The fantasy premier league API is not supported at all and there's no documentation from what I can see. I don't know if they're planning on rolling out API queries in the future, or they just abandoned the API altogether.

I wrote a script that grabs this info from the premierleague.com api and uploads to my database. You'll see in the pdf that the points are quite old, that's only because I haven't run the script in a while. You could run the script every night to update the database if you wanted.

Here's a quick example I use to display player info in a table. If I knew more Javascript I could add functionality to sort each column directly from the table. That way you could sort by team name, points etc.... rather than just player id order.

**this script grabs players with id 200 to 205 then returns their team name, photo, player name, price, points and next match. (if you want to get more player just change the initial $id variable and the for loop $id<=205 to something different.)

[PHP]<table class="playertable"id="table_id">
<thead></thead>
<tbody>
<tr bgcolor="#dbe9f2">
<th>Team</th>
<th>Photo</th>
<th>Name</th>
<th>Price</th>
<th>Points</th>
<th>Next Match</th>
</tr>
<?php
ini_set('max_execution_time', 300);
$id = 200;
for ($id=200; $id<=205; $id++)
{
?>

<tr bgcolor="#fafafa">
<?php
$endhash ='/';
$url = 'http://fantasy.premierleague.com/web/api/elements/';
$urlid = $url . $id . $endhash;
$results = file_get_contents($urlid);
$playerstats = json_decode($results, true);
$price = $playerstats;
$playerprice = number_format($price/10, 1, '.', '');
$points = $playerstats;
$webname = $playerstats;
echo "<td><img src=".$playerstats.">"." ".$playerstats."</td>";
echo "<td><img src=".$playerstats." width='45' height='45'>"."</td>";
echo "<td>".$playerstats." ".$playerstats."</td>";
echo "<td>".$playerprice."</td>";
echo "<td>".$playerstats."</td>";
echo "<td>".$playerstats."</td>";
?>
</tr>
<?php
}
?>
</table>[/PHP]

benny306

**Just realised the price in my database doesn't include decimals places. Will have to fix that.

***And there are only 593 players in the database.