1
0
mirror of https://github.com/RSS-Bridge/rss-bridge.git synced 2025-08-24 09:02:59 +02:00

Compare commits

..

70 Commits

Author SHA1 Message Date
logmanoriginal
6a98293fb3 [Configuration] Bump version to 2018-07-17 2018-07-17 20:44:01 +02:00
logmanoriginal
d79630e3b8 [Configuration] Remove check for allow_url_fopen
This commit follows the changes done in commits

fbf874cb29
ead7b2e8de
2018-07-17 20:39:14 +02:00
teromene
1f2fe25471 Fix LeBonCoinBridge, now uses getContents correctly, 2018-07-17 10:50:30 +02:00
Antoine Cadoret
87fc9e9156 fix LeBonCoin bridge (#747) 2018-07-16 20:13:08 +02:00
Nemo
c7b0c9fd31 Amazon Price Tracker Bridge (#741)
* [amazonprice] Adds AmazonPriceTracker bridge
2018-07-16 14:54:52 +02:00
Teromene
fbf874cb29 Update README.md
Remove allow_url_fopen requirement. This should no longer be necessary. Added requirement for curl.
2018-07-16 12:37:09 +02:00
Eugene Molotov
049ee52fb5 Implemented feed item categories (#746) 2018-07-16 12:32:24 +02:00
TheRadialActive
3f41d0593a Added RSS bridge for zenodo.org (#749)
* added RSS bridge for zenodo.org
2018-07-16 12:02:41 +02:00
sysadminstory
7126f5e838 [DealabsBridge] First version of the generic "Pepper" Bridge (#726)
* [DealabsBridge] First version of the generic "Pepper" Bridge
2018-07-13 00:35:13 +01:00
Nemo
ead7b2e8de [fb2] Switches to getContents (#742) 2018-07-10 02:29:47 +01:00
LogMANOriginal
0d80a19e84 [FacebookBridge] Add context for public Facebook groups (#739)
The previous context is now labeled 'User', while the new context is
labeled 'Group'. The existing code was not changed, instead new group*
functions were implemented to handle groups.

The general principle of capturing groups is the same as done for users
with adjustments to account for different HTML structures.

Captcha responses are currently not supported for groups! There doesn't
seem to be a way to trigger them consistently, which makes it hard to
handle them properly.

Features of the group context:

- The feed title is based on the group name
- The group URI used for capturing is returned for the feed URI
- Author names and timestamps are reproduced from the source
- Post titles are reproduced from the source if they exist, otherwise
the title is build manually from the author name and the content
- Original contents are included with the feed
- All images are attached as enclosures as well

Closes #
2018-07-08 17:16:00 +02:00
logmanoriginal
42c699f474 formats: Fix favicon not found if url contains path 2018-06-30 10:27:05 +02:00
logmanoriginal
2bc8daa101 [JustETFBridge] Add new bridge
Supports latest news and profiling a given ETF in Englisch, German
or Italian language. Cover images are attached as enclosures and not
as part of the content.

News:

Optionally loads the full article for each news item. Some articles
may include scripts to provide interactive graphs. These scripts are
removed as they would be rendered as pure text and a message is shown
instead: "[Content removed! Visit site to see full contents!]"

Profile:

Optionally includes the ETF strategy and description.
2018-06-30 10:27:05 +02:00
logmanoriginal
bca79d3f88 [KununuBridge] Fix broken page layout and sort reviews 2018-06-30 10:27:05 +02:00
logmanoriginal
90dc968fd1 Fix PHPCS error 2018-06-30 10:26:48 +02:00
Teromene
da6b98851c Add recuperation of the current version from git if available (#731)
* Add recuperation of the current version from git if available
* Include version when auto-reporting an error
2018-06-30 10:24:22 +02:00
teromene
71c29d4192 Fix phpcs for master. 2018-06-29 23:15:22 +01:00
LogMANOriginal
193ca87afa [phpcs] enforce single quotes (#732)
* [phpcs] Add rule to enforce single quoted strings
2018-06-29 22:55:33 +01:00
Nemo
5ea79ac1fc Add markdown support to Container Linux Feed (#730) 2018-06-28 20:54:42 +02:00
Teromene
937ea49271 Add basic authentication support (#728)
* Move configuration in its own class in order to reduce the verbosity of index.php
* Add authentication mechanism using HTTP auth
* Add a method to get the config parameters
* Remove the installation checks from the index page
* Log all failed authentication attempts
2018-06-27 19:09:41 +02:00
logmanoriginal
95686b803c [IsoHuntBridge] Remove bridge
isoHunt has discontinued services due to legal reasons and is now
accessible via https://isohunts.to

While it is certainly possible to rewrite the bridge to fetch some
information from the new site, it wouldn't be able to provide as
much functionality as before. This is due to isoHunt having removed
all searching and filtering options, only providing static HTML pages
for general categories (anime, movies, etc...). Those pages, however,
are heavily broken.

Unless someone is interested in monitoring the general categories
the effort of upgrading the bridge to the new site is not worth taking
time for.

Users of isoHunt are asked to make use of their client application,
as they don't provide online services anymore (it's now in the darknet)

Here is the statement from isoHunt:

"Due to hard regulations and security issues for bittorrent users, we
have moved into a more secure and even faster district of the internet!

[...]

Torrent Downloads have a high risk of getting legal problems. That is
why we do not offer torrentfiles any more. [...]"

-- source: https://isohunts.to
2018-06-24 18:33:50 +02:00
logmanoriginal
5087f5f79e [FacebookBridge] Support facebook links as user name
Allows users to paste facebook links as user name. The link must contain
the correct host (www.facebook.com) and a valid path (/user-name/...).
The first part of the path is used for the user name. Errors are returned
in case something went wrong.

References #706
2018-06-24 11:14:08 +02:00
logmanoriginal
4a5f190e0e [FacebookBridge] Add option to skip reviews
Reviews are provided the same way as summary posts and therefore returned
as separate feed item for each review. This commit adds a new option
'&skip_reviews=on' to skip reviews entirely.

References #706
2018-06-24 10:52:22 +02:00
logmanoriginal
01a2746715 [YoutubeBridge] Fix sniff violation
This is a fix for a sniff violation not detected by newer versions
of phpcs (not sure why though, it's detected in version 2.7.1).
2018-06-23 21:28:30 +02:00
Nemo
f4a60c1777 Add dockerfile to create an official docker image (#720) 2018-06-23 16:51:48 +02:00
sysadminstory
1b08bce779 [DealabsBridge] Follow site changes (#721)
- Changed some CSS class to follow the website changes (again)
2018-06-21 13:14:59 +01:00
Nemo
9fa74a36c6 Adds Container Linux releases RSS Feed (#718)
* Adds Container Linux releases RSS Feed
2018-06-19 19:39:08 +01:00
Corentin Garcia
7493e2b5b8 [GrandComicsDatabaseBridge] Add bridge (#717)
closes #709
2018-06-15 21:09:09 +02:00
Corentin Garcia
8e468a9ca7 [SuperSmashBlogBridge] Added bridge (#716) 2018-06-15 21:05:31 +02:00
Joe Digilio
50924b9213 Abort on parse error of config.default.ini.php (#714)
If there is an error parsing the default config file, then abort.
2018-06-15 21:02:06 +02:00
logmanoriginal
4c5013bc82 [index] Bump release version to 2018-06-10 2018-06-10 22:14:58 +02:00
Eugene Molotov
7dc09db9ca [VkBridge] More beatifications and fixes (#712)
* Add one more selector for article_author_selector
* Extend video parsing
* Add poll parsing
2018-06-10 22:09:50 +02:00
hunhejj
d92da8f0f7 Add cUrl error message and code to the debugMessage (#711) 2018-06-10 22:08:45 +02:00
logmanoriginal
064ba456e8 [InstagramBridge] Fix broken compatibility for media_type parameter
The media_type parameter was recently replaced by media_type_u (for
user mode) and media_type_h (for hashtag mode). This was necessary
in order to add the media type 'story' only for the user mode.

"The reason for that is that RSS-Bridge supports multiple parameters
with the same name if and only if they contain the exact same value.
Here, hashtags don't have stories, so it would not be possible to
pass "story" as a parameter. This is a design mistake that I made
when I added support for hashtags."

-- 8770c87389 (r28871502)

However as pointed out this change breaks existing feeds as the
parameter name is no longer compatible to previous implementations.

This commit changes the implementation to provide the old media_type
parameter globally and check for invalid options on each request. If
a user uses the 'story' option in history mode the bridge returns a
client error.

references 8770c87
references #694
fixes #696
fixes #699
fixes #701
2018-05-29 12:52:31 +02:00
LogMANOriginal
8ac8e08abf Add user config (#653)
Uses the parse_ini_file function to load default settings from the default configuration file 'config.default.ini.php'. Optionally loads custom settings from 'config.ini.php' to replace the default
values.
2018-05-29 11:52:17 +02:00
rogerdc
c4f32c31a8 Add ChristianDailyReporterBridge (#697) 2018-05-29 11:28:22 +02:00
Eugene Molotov
4369e077c2 [VkBridge] Fixed image src link generating for photo (#700) 2018-05-29 11:01:54 +02:00
sysadminstory
1045850043 [DealabsBridge] Follow site changes, fix unhandled case (#703)
* [DealabsBridge] Follow site changes, fix unhandled case

- Fixed the case where no discount was shown
- Changed some CSS class to follow the website changes
2018-05-29 10:52:13 +02:00
teromene
2d8f4dc3c5 Fix space in URL resulting in API errors. 2018-05-05 18:10:19 +01:00
teromene
779b638fb4 Added ElloBridge. Closes #683 2018-05-05 18:06:27 +01:00
teromene
3ca59392c2 Fix for crashes when accessing FileCache in case it has been purged/not created yet. 2018-05-05 18:05:48 +01:00
teromene
9b34b68180 Do not use an external service in order to fetch the favicon. 2018-05-05 13:55:38 +01:00
teromene
79ebdc4b39 Warn the user when trying to fetch a non-public facebook page. 2018-05-05 13:49:49 +01:00
teromene
8770c87389 Added support for stories in InstagramBridge. Closes #665
Renamed parameters as stories are only available in user mode.
Use a regex instead of HTML parsing to extract the JSON, as it is way faster.
2018-05-05 13:00:59 +01:00
Eugene Molotov
c1e3352218 [VkBridge] Extended article link parsing (#685)
* [VkBridge] Extended article link parsing
2018-05-05 12:03:54 +02:00
Grégory T
00570ce1b4 [ETTVBridge] New bridge, first push (#680)
* [ETTVBridge] New bridge
2018-04-30 23:18:39 +02:00
teromene
df33dcff4e [YGGTorrentBridge] URL encode the first parts of the requests. 2018-04-26 22:57:18 +01:00
Nicolas Delsaux
e60b5ab193 Mise à jour du bridge pour WorldOfTanks (#527)
* Mise à jour de l'un de mes bridges fétiches
2018-04-22 12:58:07 +02:00
teromene
b0c7a62f74 [index] Bumped version to 2018-04-20 2018-04-20 17:15:25 +02:00
teromene
57b15a089e Added DiscogsBridge. Closes #615 2018-04-20 16:57:09 +02:00
teromene
4b7fbe4188 DansTonChatBridge: test before accessing plaintext 2018-04-19 21:00:18 +02:00
Teromene
2390fb58b3 Merge pull request #673 from GregThib/patch-1
DansTonChatBridge: Update to follow DTC website changes
2018-04-19 20:58:01 +02:00
Teromene
1e8d29f6ec Merge pull request #672 from em92/patch-3
[YoutubeBridge] Removed duration in titles on search mode
2018-04-19 20:56:34 +02:00
Eugene Molotov
644d13686c [YoutubeBridge] Removed duration in titles on search mode 2018-04-19 09:03:29 +05:00
teromene
aa0ff1c9b1 Added YGGTorrentBridge. 2018-04-18 21:57:27 +02:00
teromene
539d9f1f06 Add SupInfoBridge, fixes #668 2018-04-18 12:39:45 +02:00
teromene
5ece801ce7 Fix h* display size in HtmlFormat, and fix images being wider than the page. 2018-04-18 12:29:22 +02:00
GregThib
4dcea6d9c9 Update to follow DTC website changes
Now, entry title is optionnal and may be found in h3 HTML element.
Entry content is mandatory and may be found in div[class="item-content"] HTML element.

Moreover, the title may contain simple quotes (here, encoded) so the bridge have to decode first to apply format library function. In case we don't do that, the format function double encode the quote and something like ' could appear.
2018-04-18 12:00:00 +02:00
teromene
d69e2521f1 Removed T411 bridge. Website was closed nearly one year ago. 2018-04-18 11:44:54 +02:00
teromene
7927d73719 Rewrote DemonoidBridge. Fixes #626. 2018-04-17 15:25:02 +02:00
teromene
0620f30ae0 Changed the API key used for SoundCloud bridge. Should fix #599 2018-04-17 14:24:00 +02:00
teromene
795494cfce Added enclosures to InstagramBridge. 2018-04-16 19:34:21 +02:00
teromene
ba8542156c Remove usage of function file_get_contents. 2018-04-16 19:27:20 +02:00
Eugene Molotov
55f112e034 [VkBridge] Rewrited bridge code (#667)
* [VkBridge] Convert special HTML entities to characters in pageName

* [VkBridge] Generate feed item title

* [VkBridge] Remove double backslashes in feed item link

* [VkBridge] Unpin post if pinned

* [VkBridge] Mark reposted messages

* [VkBridge] Correct external link parsing

* [VkBridge] Added article parsing

* [VkBridge] Added video parsing

* [VkBridge] Added photo parsing

* [VkBridge] Added album link parsing

* [VkBridge] Added one more external link selector

* [VkBridge] Using array of link selectors to remove

* [VkBridge] Added document parsing

* [VkBridge] Added sign parsing

* [VkBridge] Fixed incorrect sorting with pinned item

* [VkBridge] More methods to parse documents

* [VkBridge] Save fallback if page name element not found

* [VkBridge] Using post signed as feed item author

* [VkBridge] Fixed document link

* [VkBridge] Coding policy fixes
2018-04-16 10:55:31 +01:00
Mitsukarenai
208fff801d [FDroid] minor fixes for Travis CI 2018-04-15 13:21:48 +02:00
Mitsukarenai
3c9860de43 [FDroid] new bridge 2018-04-15 13:13:10 +02:00
Adam Tygart
a16ec196c5 [NotAlways] Add a bridge for the NotAlways family of sites (#537)
NotAlways right found it necessary to remove their RSS feeds recently. This is a *simple* bridge to grab the ones on the front page. It allows you to filter the articles based on their classification (right, working, romantic, related, learning, friendly, hopeless, unfiltered, or all).
2018-04-15 12:02:37 +01:00
teromene
887fc7b037 Fix GoComics, website completely changed. Fixes #663 2018-04-14 18:15:44 +01:00
teromene
1bd4a40f71 Added GNOME Builder configuration to gitignore. 2018-04-14 18:15:13 +01:00
teromene
494169f959 Added bridge for Pixiv.
This bridge is slow, as caching of images is required (REFERER header required to access the full size images)
2018-04-14 16:19:35 +01:00
76 changed files with 4710 additions and 1254 deletions

8
.dockerignore Normal file
View File

@@ -0,0 +1,8 @@
.git
cache/*
DEBUG
Dockerfile
whitelist.txt
phpcs.xml
CHANGELOG.md
CONTRIBUTING.md

6
.gitignore vendored
View File

@@ -227,8 +227,12 @@ pip-log.txt
/cache
/whitelist.txt
DEBUG
config.ini.php
######################
## VisualStudioCode ##
######################
.vscode/*
.vscode/*
#Builder
.buildconfig

5
Dockerfile Normal file
View File

@@ -0,0 +1,5 @@
FROM ulsmith/alpine-apache-php7
COPY ./ /app/public/
RUN chown -R apache:root /app/public

View File

@@ -53,7 +53,7 @@ Requirements
* PHP 5.6, e.g. `AddHandler application/x-httpd-php56 .php` in `.htaccess`
* `openssl` extension enabled in PHP config (`php.ini`)
* `allow_url_fopen=1` in `php.ini`
* `curl` extension enabled in PHP config (`php.ini`)
Enabling/Disabling bridges
===

View File

@@ -0,0 +1,149 @@
<?php
class AmazonPriceTrackerBridge extends BridgeAbstract {
const MAINTAINER = 'captn3m0';
const NAME = 'Amazon Price Tracker';
const URI = 'https://www.amazon.com/';
const CACHE_TIMEOUT = 3600; // 1h
const DESCRIPTION = 'Tracks price for a single product on Amazon';
const PARAMETERS = array(
array(
'asin' => array(
'name' => 'ASIN',
'required' => true,
'exampleValue' => 'B071GB1VMQ',
// https://stackoverflow.com/a/12827734
'pattern' => 'B[\dA-Z]{9}|\d{9}(X|\d)',
),
'tld' => array(
'name' => 'Country',
'type' => 'list',
'required' => true,
'values' => array(
'Australia' => 'com.au',
'Brazil' => 'com.br',
'Canada' => 'ca',
'China' => 'cn',
'France' => 'fr',
'Germany' => 'de',
'India' => 'in',
'Italy' => 'it',
'Japan' => 'co.jp',
'Mexico' => 'com.mx',
'Netherlands' => 'nl',
'Spain' => 'es',
'United Kingdom' => 'co.uk',
'United States' => 'com',
),
'defaultValue' => 'com',
),
));
protected $title;
/**
* Generates domain name given a amazon TLD
*/
private function getDomainName() {
return 'https://www.amazon.' . $this->getInput('tld');
}
/**
* Generates URI for a Amazon product page
*/
public function getURI() {
if (!is_null($this->getInput('asin'))) {
return $this->getDomainName() . '/dp/' . $this->getInput('asin') . '/';
}
return parent::getURI();
}
/**
* Scrapes the product title from the html page
* returns the default title if scraping fails
*/
private function getTitle($html) {
$titleTag = $html->find('#productTitle', 0);
if (!$titleTag) {
return $this->getDefaultTitle();
} else {
return trim(html_entity_decode($titleTag->innertext, ENT_QUOTES));
}
}
/**
* Title used by the feed if none could be found
*/
private function getDefaultTitle() {
return 'Amazon.' . $this->getInput('tld') . ': ' . $this->getInput('asin');
}
/**
* Returns name for the feed
* Uses title (already scraped) if it has one
*/
public function getName() {
if (isset($this->title)) {
return $this->title;
} else {
return parent::getName();
}
}
/**
* Returns a generated image tag for the product
*/
private function getImage($html) {
$imageSrc = $html->find('#main-image-container img', 0);
if ($imageSrc) {
$imageSrc = $imageSrc ? $imageSrc->getAttribute('data-old-hires') : '';
return <<<EOT
<img width="300" style="max-width:300;max-height:300" src="$imageSrc" alt="{$this->title}" />
EOT;
}
}
/**
* Return \simple_html_dom object
* for the entire html of the product page
*/
private function getHtml() {
$uri = $this->getURI();
return getSimpleHTMLDOM($uri) ?: returnServerError('Could not request Amazon.');
}
/**
* Scrape method for Amazon product page
* @return [type] [description]
*/
public function collectData() {
$html = $this->getHtml();
$this->title = $this->getTitle($html);
$imageTag = $this->getImage($html);
$asinData = $html->find('#cerberus-data-metrics', 0);
// <div id="cerberus-data-metrics" style="display: none;"
// data-asin="B00WTHJ5SU" data-asin-price="14.99" data-asin-shipping="0"
// data-asin-currency-code="USD" data-substitute-count="-1" ... />
$currency = $asinData->getAttribute('data-asin-currency-code');
$shipping = $asinData->getAttribute('data-asin-shipping');
$price = $asinData->getAttribute('data-asin-price');
$item = array(
'title' => $this->title,
'uri' => $this->getURI(),
'content' => "$imageTag<br/>Price: $price $currency",
);
if ($shipping !== '0') {
$item['content'] .= "<br>Shipping: $shipping $currency</br>";
}
$this->items[] = $item;
}
}

View File

@@ -19,7 +19,7 @@ class BlaguesDeMerdeBridge extends BridgeAbstract {
$item['content'] = trim($element->find('div.joke_text_contener', 0)->innertext);
$uri = $temp[2]->href;
$item['uri'] = $uri;
$item['title'] = substr($uri, (strrpos($uri, "/") + 1));
$item['title'] = substr($uri, (strrpos($uri, '/') + 1));
$date = $element->find('li.bdm_date', 0)->innertext;
$time = mktime(0, 0, 0, substr($date, 3, 2), substr($date, 0, 2), substr($date, 6, 4));
$item['timestamp'] = $time;

View File

@@ -23,14 +23,14 @@ class CADBridge extends FeedExpander {
if($html3 == false)
return 'Daily comic not released yet';
$htmlpart = explode("/", $url);
$htmlpart = explode('/', $url);
switch ($htmlpart[3]) {
case 'cad':
preg_match_all("/http:\/\/cdn2\.cad-comic\.com\/comics\/cad-\S*png/", $html3, $url2);
preg_match_all('/http:\/\/cdn2\.cad-comic\.com\/comics\/cad-\S*png/', $html3, $url2);
break;
case 'sillies':
preg_match_all("/http:\/\/cdn2\.cad-comic\.com\/comics\/sillies-\S*gif/", $html3, $url2);
preg_match_all('/http:\/\/cdn2\.cad-comic\.com\/comics\/sillies-\S*gif/', $html3, $url2);
break;
default:
return 'Daily comic not released yet';

View File

@@ -0,0 +1,25 @@
<?php
class ChristianDailyReporterBridge extends BridgeAbstract {
const MAINTAINER = 'rogerdc';
const NAME = 'Christian Daily Reporter Unofficial RSS';
const URI = 'https://www.christiandailyreporter.com/';
const DESCRIPTION = 'The Unofficial Christian Daily Reporter RSS';
// const CACHE_TIMEOUT = 86400; // 1 day
public function collectData() {
$uri = 'https://www.christiandailyreporter.com/';
$html = getSimpleHTMLDOM($uri)
or returnServerError('Could not request Christian Daily Reporter.');
foreach($html->find('div.top p a,div.column p a') as $element) {
$item = array();
// Title
$item['title'] = $element->innertext;
// URL
$item['uri'] = $element->href;
$this->items[] = $item;
}
}
}

View File

@@ -0,0 +1,93 @@
<?php
class ContainerLinuxReleasesBridge extends BridgeAbstract {
const MAINTAINER = 'captn3m0';
const NAME = 'Core OS Container Linux Releases Bridge';
const URI = 'https://coreos.com/releases/';
const DESCRIPTION = 'Returns the releases notes for Container Linux';
const STABLE = 'stable';
const BETA = 'beta';
const ALPHA = 'alpha';
const PARAMETERS = [
[
'channel' => [
'name' => 'Release Channel',
'type' => 'list',
'required' => true,
'defaultValue' => self::STABLE,
'values' => [
'Stable' => self::STABLE,
'Beta' => self::BETA,
'Alpha' => self::ALPHA,
],
]
]
];
public function getReleaseFeed($jsonUrl) {
$json = getContents($jsonUrl)
or returnServerError('Could not request Core OS Website.');
return json_decode($json, true);
}
public function collectData() {
$data = $this->getReleaseFeed($this->getJsonUri());
foreach ($data as $releaseVersion => $release) {
$item = [];
$item['uri'] = "https://coreos.com/releases/#$releaseVersion";
$item['title'] = $releaseVersion;
$content = $release['release_notes'];
$content .= <<<EOT
Major Software:
* Kernel: {$release['major_software']['kernel'][0]}
* Docker: {$release['major_software']['docker'][0]}
* etcd: {$release['major_software']['etcd'][0]}
EOT;
$item['timestamp'] = strtotime($release['release_date']);
// Based on https://gist.github.com/jbroadway/2836900
// Links
$regex = '/\[([^\[]+)\]\(([^\)]+)\)/';
$replacement = '<a href=\'\2\'>\1</a>';
$item['content'] = preg_replace($regex, $replacement, $content);
// Headings
$regex = '/^(.*)\:\s?$/m';
$replacement = '<h3>\1</h3>';
$item['content'] = preg_replace($regex, $replacement, $item['content']);
// Lists
$regex = '/\n\s*[\*|\-](.*)/';
$item['content'] = preg_replace_callback ($regex, function($regs) {
$item = $regs[1];
return sprintf ('<ul><li>%s</li></ul>', trim ($item));
}, $item['content']);
$this->items[] = $item;
}
}
private function getJsonUri() {
$channel = $this->getInput('channel');
return "https://coreos.com/releases/releases-$channel.json";
}
public function getURI() {
return self::URI;
}
public function getName(){
if(!is_null($this->getInput('channel'))) {
return 'Container Linux Releases: ' . $this->getInput('channel') . ' Channel';
}
return parent::getName();
}
}

View File

@@ -25,7 +25,7 @@ class CopieDoubleBridge extends BridgeAbstract {
} elseif(strpos($element->innertext, '/images/suivant.gif') === false) {
$a = $element->find('a', 0);
$item['uri'] = self::URI . $a->href;
$content = str_replace('src="/', 'src="/' . self::URI, $element->find("td", 0)->innertext);
$content = str_replace('src="/', 'src="/' . self::URI, $element->find('td', 0)->innertext);
$content = str_replace('href="/', 'href="' . self::URI, $content);
$item['content'] = $content;
$this->items[] = $item;

View File

@@ -11,7 +11,7 @@ class CourrierInternationalBridge extends BridgeAbstract {
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Error.');
$element = $html->find("article");
$element = $html->find('article');
$article_count = 1;
foreach($element as $article) {

View File

@@ -16,7 +16,7 @@ class CpasbienBridge extends BridgeAbstract {
));
public function collectData(){
$request = str_replace(" ", "-", trim($this->getInput('q')));
$request = str_replace(' ', '-', trim($this->getInput('q')));
$html = getSimpleHTMLDOM(self::URI . '/recherche/' . urlencode($request) . '.html')
or returnServerError('No results for this query.');

View File

@@ -41,7 +41,7 @@ class DanbooruBridge extends BridgeAbstract {
$item = array();
$item['uri'] = $element->find('a', 0)->href;
$item['postid'] = (int)preg_replace("/[^0-9]/", '', $element->getAttribute(static::IDATTRIBUTE));
$item['postid'] = (int)preg_replace('/[^0-9]/', '', $element->getAttribute(static::IDATTRIBUTE));
$item['timestamp'] = time();
$thumbnailUri = $element->find('img', 0)->src;
$item['tags'] = $this->getTags($element);

View File

@@ -15,8 +15,13 @@ class DansTonChatBridge extends BridgeAbstract {
foreach($html->find('div.item') as $element) {
$item = array();
$item['uri'] = $element->find('a', 0)->href;
$item['title'] = 'DansTonChat ' . $element->find('a', 1)->plaintext;
$item['content'] = $element->find('a', 0)->innertext;
$titleContent = $element->find('h3 a', 0);
if($titleContent) {
$item['title'] = 'DansTonChat ' . html_entity_decode($titleContent->plaintext, ENT_QUOTES);
} else {
$item['title'] = 'DansTonChat';
}
$item['content'] = $element->find('div.item-content a', 0)->innertext;
$this->items[] = $item;
}
}

View File

@@ -1,8 +1,9 @@
<?php
class DealabsBridge extends BridgeAbstract {
const NAME = 'Dealabs search bridge';
class DealabsBridge extends PepperBridgeAbstract {
const NAME = 'Dealabs Bridge';
const URI = 'https://www.dealabs.com/';
const DESCRIPTION = 'Return the Dealabs search result using keywords';
const DESCRIPTION = 'Affiche les Deals de Dealabs';
const MAINTAINER = 'sysadminstory';
const PARAMETERS = array(
'Recherche par Mot(s) clé(s)' => array (
@@ -39,7 +40,7 @@ class DealabsBridge extends BridgeAbstract {
),
'Deals par groupe' => array(
'groupe' => array(
'group' => array(
'name' => 'Groupe',
'type' => 'list',
'required' => 'true',
@@ -61,10 +62,10 @@ class DealabsBridge extends BridgeAbstract {
'Services divers' => 'services-divers',
'Sports & plein air' => 'sports-plein-air',
'Téléphonie' => 'telephonie',
'Voyages & sorties' => 'voyages-sorties-restaurants'
'Voyages & sorties' => 'voyages-sorties-restaurants',
)
),
'ordre' => array(
'order' => array(
'name' => 'Trier par',
'type' => 'list',
'required' => 'true',
@@ -78,37 +79,99 @@ class DealabsBridge extends BridgeAbstract {
)
);
public $lang = array(
'bridge-uri' => SELF::URI,
'bridge-name' => SELF::NAME,
'context-keyword' => 'Recherche par Mot(s) clé(s)',
'context-group' => 'Deals par groupe',
'uri-group' => '/groupe/',
'request-error' => 'Could not request Dealabs',
'no-results' => 'Il n&#039;y a rien à afficher pour le moment :(',
'relative-date-indicator' => array(
'il y a',
),
'price' => 'Prix',
'shipping' => 'Livraison',
'origin' => 'Origine',
'discount' => 'Réduction',
'title-keyword' => 'Recherche',
'title-group' => 'Groupe',
'local-months' => array(
'janvier',
'février',
'mars',
'avril',
'mai',
'juin',
'juillet',
'août',
'septembre',
'octobre',
'novembre',
'décembre'
),
'local-time-relative' => array(
'il y a ',
'min',
'h',
'jour',
'jours',
'mois',
'ans',
'et '
),
'date-prefixes' => array(
'Actualisé ',
),
'relative-date-alt-prefixes' => array(
'Actualisé ',
),
'relative-date-ignore-suffix' => array(
),
'localdeal' => array(
'Local',
'Pays d\'expédition'
),
);
}
class PepperBridgeAbstract extends BridgeAbstract {
const CACHE_TIMEOUT = 3600;
public function collectData(){
switch($this->queriedContext) {
case 'Recherche par Mot(s) clé(s)':
return $this->collectDataMotsCles();
case $this->i8n('context-keyword'):
return $this->collectDataKeywords();
break;
case 'Deals par groupe':
return $this->collectDataGroupe();
case $this->i8n('context-group'):
return $this->collectDataGroup();
break;
}
}
/**
* Get the Deal data from the choosen groupe in the choose order
* Get the Deal data from the choosen group in the choosed order
*/
public function collectDataGroupe()
public function collectDataGroup()
{
$groupe = $this->getInput('groupe');
$ordre = $this->getInput('ordre');
$group = $this->getInput('group');
$order = $this->getInput('order');
$url = self::URI
. '/groupe/' . $groupe . $ordre;
$url = $this->i8n('bridge-uri')
. $this->i8n('uri-group') . $group . $order;
$this->collectDeals($url);
}
/**
* Get the Deal data from the choosen keywords and parameters
*/
public function collectDataMotsCles()
public function collectDataKeywords()
{
$q = $this->getInput('q');
$hide_expired = $this->getInput('hide_expired');
@@ -117,7 +180,7 @@ class DealabsBridge extends BridgeAbstract {
$priceTo = $this->getInput('priceFrom');
/* Even if the original website uses POST with the search page, GET works too */
$url = self::URI
$url = $this->i8n('bridge-uri')
. '/search/advanced?q='
. urlencode($q)
. '&hide_expired='. $hide_expired
@@ -138,8 +201,8 @@ class DealabsBridge extends BridgeAbstract {
*/
public function collectDeals($url){
$html = getSimpleHTMLDOM($url)
or returnServerError('Could not request Dealabs.');
$list = $html->find('article');
or returnServerError($this->i8n('request-error'));
$list = $html->find('article[id]');
// Deal Image Link CSS Selector
$selectorImageLink = implode(
@@ -148,7 +211,6 @@ class DealabsBridge extends BridgeAbstract {
'cept-thread-image-link',
'imgFrame',
'imgFrame--noBorder',
'box--all-i',
'thread-listImgCell',
)
);
@@ -181,7 +243,7 @@ class DealabsBridge extends BridgeAbstract {
'cept-description-container',
'overflow--wrap-break',
'size--all-s',
'size--fromW3-m',
'size--fromW3-m'
)
);
@@ -191,7 +253,6 @@ class DealabsBridge extends BridgeAbstract {
array(
'size--all-s',
'flex',
'flex--wrap',
'flex--justify-e',
'flex--grow-1',
)
@@ -199,40 +260,44 @@ class DealabsBridge extends BridgeAbstract {
// If there is no results, we don't parse the content because it display some random deals
$noresult = $html->find('h3[class=size--all-l size--fromW2-xl size--fromW3-xxl]', 0);
if($noresult != null && $noresult->plaintext == 'Il n&#039;y a rien à afficher pour le moment :(') {
if ($noresult != null && strpos($noresult->plaintext, $this->i8n('no-results')) !== false) {
$this->items = array();
} else {
foreach($list as $deal) {
foreach ($list as $deal) {
$item = array();
$item['uri'] = $deal->find('div[class=threadGrid-title]', 0)->find('a', 0)->href;
$item['title'] = $deal->find('a[class*='. $selectorLink .']', 0
)->plaintext;
)->plaintext;
$item['author'] = $deal->find('span.thread-username', 0)->plaintext;
$item['content'] = '<table><tr><td><a href="'
. $deal->find(
'a[class*='. $selectorImageLink .']', 0)->href
. '"><img src="'
. $this->getImage($deal)
. '"/></td><td><h2><a href="'
. $deal->find('a[class*='. $selectorLink .']', 0)->href
. '">'
. $deal->find('a[class*='. $selectorLink .']', 0)->innertext
. '</a></h2>'
. $this->getPrix($deal)
. $this->getReduction($deal)
. $this->getExpedition($deal)
. $this->getLivraison($deal)
. $this->getOrigine($deal)
. $deal->find('div[class='. $selectorDescription .']', 0)->innertext
. '</td><td>'
. $deal->find('div[class='. $selectorHot .']', 0)->children(0)->outertext
. '</td></table>';
$dealDateDiv = $deal->find('div[class='. $selectorDate .']', 0)
. '"><img src="'
. $this->getImage($deal)
. '"/></td><td><h2><a href="'
. $deal->find('a[class*='. $selectorLink .']', 0)->href
. '">'
. $deal->find('a[class*='. $selectorLink .']', 0)->innertext
. '</a></h2>'
. $this->getPrice($deal)
. $this->getDiscount($deal)
. $this->getShipsFrom($deal)
. $this->getShippingCost($deal)
. $this->GetSource($deal)
. $deal->find('div[class*='. $selectorDescription .']', 0)->innertext
. '</td><td>'
. $deal->find('div[class='. $selectorHot .']', 0)->children(0)->outertext
. '</td></table>';
$dealDateDiv = $deal->find('div[class*='. $selectorDate .']', 0)
->find('span[class=hide--toW3]');
$itemDate = end($dealDateDiv)->plaintext;
if(substr( $itemDate, 0, 6 ) === 'il y a') {
// In case of a Local deal, there is no date, but we can use
// this case for other reason (like date not in the last field)
if ($this->contains($itemDate, $this->i8n('localdeal'))) {
$item['timestamp'] = time();
} else if ($this->contains($itemDate, $this->i8n('relative-date-indicator'))) {
$item['timestamp'] = $this->relativeDateToTimestamp($itemDate);
} else {
} else {
$item['timestamp'] = $this->parseDate($itemDate);
}
$this->items[] = $item;
@@ -240,15 +305,29 @@ class DealabsBridge extends BridgeAbstract {
}
}
/**
* Check if the string $str contains any of the string of the array $arr
* @return boolean true if the string matched anything otherwise false
*/
private function contains($str, array $arr)
{
foreach ($arr as $a) {
if (stripos($str, $a) !== false) {
return true;
}
}
return false;
}
/**
* Get the Price from a Deal if it exists
* @return string String of the deal price
*/
private function getPrix($deal)
private function getPrice($deal)
{
if($deal->find(
if ($deal->find(
'span[class*=thread-price]', 0) != null) {
return '<div>Prix : '
return '<div>'.$this->i8n('price') .' : '
. $deal->find(
'span[class*=thread-price]', 0
)->plaintext
@@ -263,17 +342,17 @@ class DealabsBridge extends BridgeAbstract {
* Get the Shipping costs from a Deal if it exists
* @return string String of the deal shipping Cost
*/
private function getLivraison($deal)
private function getShippingCost($deal)
{
if($deal->find('span[class*=cept-shipping-price]', 0) != null) {
if($deal->find('span[class*=cept-shipping-price]', 0)->children(0) != null) {
return '<div>Livraison : '
. $deal->find('span[class*=cept-shipping-price]', 0)->children(0)->innertext
. '</div>';
if ($deal->find('span[class*=cept-shipping-price]', 0) != null) {
if ($deal->find('span[class*=cept-shipping-price]', 0)->children(0) != null) {
return '<div>'. $this->i8n('shipping') .' : '
. $deal->find('span[class*=cept-shipping-price]', 0)->children(0)->innertext
. '</div>';
} else {
return '<div>Livraison : '
. $deal->find('span[class*=cept-shipping-price]', 0)->innertext
. '</div>';
return '<div>'. $this->i8n('shipping') .' : '
. $deal->find('span[class*=cept-shipping-price]', 0)->innertext
. '</div>';
}
} else {
return '';
@@ -284,10 +363,10 @@ class DealabsBridge extends BridgeAbstract {
* Get the source of a Deal if it exists
* @return string String of the deal source
*/
private function getOrigine($deal)
private function GetSource($deal)
{
if($deal->find('a[class=text--color-greyShade]', 0) != null) {
return '<div>Origine : '
if ($deal->find('a[class=text--color-greyShade]', 0) != null) {
return '<div>'. $this->i8n('origin') .' : '
. $deal->find('a[class=text--color-greyShade]', 0)->outertext
. '</div>';
} else {
@@ -299,15 +378,21 @@ class DealabsBridge extends BridgeAbstract {
* Get the original Price and discout from a Deal if it exists
* @return string String of the deal original price and discount
*/
private function getReduction($deal)
private function getDiscount($deal)
{
if($deal->find('span[class*=mute--text text--lineThrough]', 0) != null) {
return '<div>Réduction : <span style="text-decoration: line-through;">'
if ($deal->find('span[class*=mute--text text--lineThrough]', 0) != null) {
$discountHtml = $deal->find('span[class=space--ml-1 size--all-l size--fromW3-xl]', 0);
if ($discountHtml != null) {
$discount = $discountHtml->plaintext;
} else {
$discount = '';
}
return '<div>'. $this->i8n('discount') .' : <span style="text-decoration: line-through;">'
. $deal->find(
'span[class*=mute--text text--lineThrough]', 0
)->plaintext
)->plaintext
. '</span>&nbsp;'
. $deal->find('span[class=space--ml-1 size--all-l size--fromW3-xl]', 0)->plaintext
. $discount
. '</div>';
} else {
return '';
@@ -320,7 +405,6 @@ class DealabsBridge extends BridgeAbstract {
*/
private function getImage($deal)
{
$selectorLazy = implode(
' ', /* Notice this is a space! */
array(
@@ -334,7 +418,7 @@ class DealabsBridge extends BridgeAbstract {
)
);
$selectorPlain = implode(
$selectorPlain = implode(
' ', /* Notice this is a space! */
array(
'thread-image',
@@ -344,21 +428,21 @@ class DealabsBridge extends BridgeAbstract {
'cept-thread-img'
)
);
if($deal->find('img[class='. $selectorLazy .']', 0) != null) {
if ($deal->find('img[class='. $selectorLazy .']', 0) != null) {
return json_decode(
html_entity_decode(
$deal->find('img[class='. $selectorLazy .']', 0)
->getAttribute('data-lazy-img')))->{'src'};
->getAttribute('data-lazy-img')))->{'src'};
} else {
return $deal->find('img[class='. $selectorPlain .']', 0 )->src;
return $deal->find('img[class*='. $selectorPlain .']', 0 )->src;
}
}
/**
* Get the originating country from a Deal if it existsa
* Get the originating country from a Deal if it exists
* @return string String of the deal originating country
*/
private function getExpedition($deal)
private function getShipsFrom($deal)
{
$selector = implode(
' ', /* Notice this is a space! */
@@ -369,7 +453,7 @@ class DealabsBridge extends BridgeAbstract {
'text--color-greyShade'
)
);
if($deal->find('span[class='. $selector .']', 0) != null) {
if ($deal->find('span[class='. $selector .']', 0) != null) {
return '<div>'
. $deal->find('span[class='. $selector .']', 0)->children(2)->plaintext
. '</div>';
@@ -379,25 +463,12 @@ class DealabsBridge extends BridgeAbstract {
}
/**
* Transforms a French date into a timestam
* Transforms a local date into a timestamp
* @return int timestamp of the input date
*/
private function parseDate($string)
{
$month_fr = array(
'janvier',
'février',
'mars',
'avril',
'mai',
'juin',
'juillet',
'août',
'septembre',
'octobre',
'novembre',
'décembre'
);
$month_local = $this->i8n('local-months');
$month_en = array(
'January',
'February',
@@ -412,11 +483,18 @@ class DealabsBridge extends BridgeAbstract {
'November',
'December'
);
$date_str = trim(str_replace($month_fr, $month_en, $string));
if(!preg_match('/[0-9]{4}/', $string)) {
// A date can be prfixed with some words, we remove theme
$string = $this->removeDatePrefixes($string);
// We translate the local months name in the english one
$date_str = trim(str_replace($month_local, $month_en, $string));
// If the date does not contain any year, we add the current year
if (!preg_match('/[0-9]{4}/', $string)) {
$date_str .= ' ' . date('Y');
}
// Add the Hour and minutes
$date_str .= ' 00:00';
$date = DateTime::createFromFormat('j F Y H:i', $date_str);
@@ -424,21 +502,41 @@ class DealabsBridge extends BridgeAbstract {
}
/**
* Transforms a relate French date into a timestam
* Remove the prefix of a date if it has one
* @return the date without prefiux
*/
private function removeDatePrefixes($string)
{
$string = str_replace($this->i8n('date-prefixes'), array(), $string);
return $string;
}
/**
* Remove the suffix of a relative date if it has one
* @return the relative date without suffixes
*/
private function removeRelativeDateSuffixes($string)
{
if (count($this->i8n('relative-date-ignore-suffix')) > 0) {
$string = preg_replace($this->i8n('relative-date-ignore-suffix'), '', $string);
}
return $string;
}
/**
* Transforms a relative local date into a timestamp
* @return int timestamp of the input date
*/
private function relativeDateToTimestamp($str) {
$date = new DateTime();
$search = array(
'il y a ',
'min',
'h',
'jour',
'jours',
'mois',
'ans',
'et '
);
// In case of update date, replace it by the regular relative date first word
$str = str_replace($this->i8n('relative-date-alt-prefixes'), $this->i8n('local-time-relative')[0], $str);
$str = $this->removeRelativeDateSuffixes($str);
$search = $this->i8n('local-time-relative');
$replace = array(
'-',
'minute',
@@ -453,18 +551,38 @@ class DealabsBridge extends BridgeAbstract {
return $date->getTimestamp();
}
/**
* Returns the RSS Feed title according to the parameters
* @return string the RSS feed Tiyle
*/
public function getName(){
switch($this->queriedContext) {
case 'Recherche par Mot(s) clé(s)':
return self::NAME . ' - Recherche : '. $this->getInput('q');
case $this->i8n('context-keyword'):
return $this->i8n('bridge-name') . ' - '. $this->i8n('title-keyword') .' : '. $this->getInput('q');
break;
case 'Deals par groupe':
$values = self::PARAMETERS['Deals par groupe']['groupe']['values'];
$groupe = array_search($this->getInput('groupe'), $values);
return self::NAME . ' - Groupe : '. $groupe;
case $this->i8n('context-group'):
$values = $this->getParameters()[$this->i8n('context-group')]['group']['values'];
$group = array_search($this->getInput('group'), $values);
return $this->i8n('bridge-name') . ' - '. $this->i8n('title-group'). ' : '. $group;
break;
default: // Return default value
return self::NAME;
return static::NAME;
}
}
/**
* This is some "localisation" function that returns the needed content using
* the "$lang" class variable in the local class
* @return various the local content needed
*/
public function i8n($key)
{
if (array_key_exists($key, $this->lang)) {
return $this->lang[$key];
} else {
return null;
}
}

View File

@@ -35,11 +35,11 @@ class DemoBridge extends BridgeAbstract {
public function collectData(){
$item = array();
$item['author'] = "Me!";
$item['title'] = "Test";
$item['content'] = "Awesome content !";
$item['id'] = "Lalala";
$item['uri'] = "http://example.com/test";
$item['author'] = 'Me!';
$item['title'] = 'Test';
$item['content'] = 'Awesome content !';
$item['id'] = 'Lalala';
$item['uri'] = 'http://example.com/test';
$this->items[] = $item;
}

View File

@@ -4,143 +4,163 @@ class DemonoidBridge extends BridgeAbstract {
const MAINTAINER = 'metaMMA';
const NAME = 'Demonoid';
const URI = 'https://www.demonoid.pw/';
const DESCRIPTION = 'Returns results for the keywords (in all categories or
a specific category). You can put several keywords separated by a semicolon
(e.g. "one show;another show"). Searches can by done in a specific category;
category number must be specified. (All=0, Movies=1, Music=2, TV=3, Games=4,
Applications=5, Pictures=8, Anime=9, Comics=10, Books=11 Music Videos=8,
Audio Books=17). User feed takes the Uploader ID number (not uploader name)
as keyword. Uploader ID is found by clicking on uploader, clicking on
"View this user\'s torrents", and copying the number after "uid=". An entire
category feed is accomplished by leaving "keywords" box blank and using the
corresponding category number.';
const DESCRIPTION = 'Returns results from search';
const PARAMETERS = array( array(
const PARAMETERS = array(array(
'q' => array(
'name' => 'keywords/user ID/category, separated by semicolons',
'exampleValue' => 'first list;second list;…',
'required' => true
),
'crit' => array(
'name' => 'keywords',
'exampleValue' => 'keyword1 keyword2…',
'required' => true,
),
'category' => array(
'name' => 'Category',
'type' => 'list',
'name' => 'Feed type',
'values' => array(
'search' => 'search',
'category' => 'cat',
'user' => 'usr'
'All' => 0,
'Movies' => 1,
'Music' => 2,
'TV' => 3,
'Games' => 4,
'Applications' => 5,
'Pictures' => 8,
'Anime' => 9,
'Comics' => 10,
'Books' => 11,
'Audiobooks' => 17
)
)
),
'catCheck' => array(
'type' => 'checkbox',
'name' => 'Specify category for keyword search ?',
),
'cat' => array(
'name' => 'Category number',
),
));
), array(
'catOnly' => array(
'name' => 'Category',
'type' => 'list',
'values' => array(
'All' => 0,
'Movies' => 1,
'Music' => 2,
'TV' => 3,
'Games' => 4,
'Applications' => 5,
'Pictures' => 8,
'Anime' => 9,
'Comics' => 10,
'Books' => 11,
'Audiobooks' => 17
)
)
), array(
'userid' => array(
'name' => 'user id',
'exampleValue' => '00000',
'required' => true,
'type' => 'number'
),
'category' => array(
'name' => 'Category',
'type' => 'list',
'values' => array(
'All' => 0,
'Movies' => 1,
'Music' => 2,
'TV' => 3,
'Games' => 4,
'Applications' => 5,
'Pictures' => 8,
'Anime' => 9,
'Comics' => 10,
'Books' => 11,
'Audiobooks' => 17
)
)
)
);
public function collectData() {
$catBool = $this->getInput('catCheck');
if($catBool) {
$catNum = $this->getInput('cat');
if(!empty($this->getInput('q'))) {
$html = getSimpleHTMLDOM(
self::URI .
'files/?category=' .
rawurlencode($this->getInput('category')) .
'&subcategory=All&quality=All&seeded=2&external=2&query=' .
urlencode($this->getInput('q')) .
'&uid=0&sort='
) or returnServerError('Could not request Demonoid.');
} elseif(!empty($this->getInput('catOnly'))) {
$html = getSimpleHTMLDOM(
self::URI .
'files/?uid=0&category=' .
rawurlencode($this->getInput('catOnly')) .
'&subcategory=0&language=0&seeded=2&quality=0&query=&sort='
) or returnServerError('Could not request Demonoid.');
} elseif(!empty($this->getInput('userid'))) {
$html = getSimpleHTMLDOM(
self::URI .
'files/?uid=' .
rawurlencode($this->getInput('userid')) .
'&seeded=2'
) or returnServerError('Could not request Demonoid.');
} else {
returnServerError('Invalid parameters !');
}
$critList = $this->getInput('crit');
$keywordsList = explode(';', $this->getInput('q'));
foreach($keywordsList as $keywords) {
switch($critList) {
case 'search':
if($catBool == false) {
$html = file_get_contents(
self::URI .
'files/?category=0&subcategory=All&quality=All&seeded=2&external=2&query=' .
urlencode($keywords) . #not rawurlencode so space -> '+'
'&uid=0&sort='
) or returnServerError('Could not request Demonoid.');
} else {
$html = file_get_contents(
self::URI .
'files/?category=' .
rawurlencode($catNum) .
'&subcategory=All&quality=All&seeded=2&external=2&query=' .
urlencode($keywords) . #not rawurlencode so space -> '+'
'&uid=0&sort='
) or returnServerError('Could not request Demonoid.');
}
break;
case 'usr':
$html = file_get_contents(
self::URI .
'files/?uid=' .
rawurlencode($keywords) .
'&seeded=2'
) or returnServerError('Could not request Demonoid.');
break;
case 'cat':
$html = file_get_contents(
self::URI .
'files/?uid=0&category=' .
rawurlencode($keywords) .
'&subcategory=0&language=0&seeded=2&quality=0&query=&sort='
) or returnServerError('Could not request Demonoid.');
break;
if(preg_match('~No torrents found~', $html)) {
return;
}
$table = $html->find('td[class=ctable_content_no_pad]', 0);
$cursorCount = 4;
$elementCount = 0;
while($elementCount != 40) {
$elementCount++;
$currentElement = $table->find('tr', $cursorCount);
if(preg_match('~items total~', $currentElement)) {
break;
}
if(preg_match('~No torrents found~', $html)) {
returnServerError('No result for query ' . $keywords);
}
$bigTable = explode('<!-- start torrent list -->', $html)[1];
$last50 = explode('<!-- end torrent list -->', $bigTable)[0];
$dateChunk = explode('added_today', $last50);
$item = array ();
for($block = 1;$block < count($dateChunk);$block++) {
preg_match('~(?<=>Add).*?(?=<)~', $dateChunk[$block], $dateStr);
$item = array();
//Do we have a date ?
if(preg_match('~Added.*?(.*)~', $currentElement->plaintext, $dateStr)) {
if(preg_match('~today~', $dateStr[0])) {
date_default_timezone_set('UTC');
$timestamp = mktime(0, 0, 0, gmdate('n'), gmdate('j'), gmdate('Y'));
} else {
preg_match('~(?<=ed on ).*\d+~', $dateStr[0], $fullDateStr);
} else {
preg_match('~(?<=ed on ).*\d+~', $currentElement->plaintext, $fullDateStr);
date_default_timezone_set('UTC');
$dateObj = strptime($fullDateStr[0], '%A, %b %d, %Y');
$timestamp = mktime(0, 0, 0, $dateObj['tm_mon'] + 1, $dateObj['tm_mday'], 1900 + $dateObj['tm_year']);
}
$itemsChunk = explode('<!-- tstart -->', $dateChunk[$block]);
for($items = 1;$items < count($itemsChunk);$items++) {
$item = array();
$cols = explode('<td', $itemsChunk[$items]);
preg_match('~(?<=href=\"/).*?(?=\")~', $cols[1], $matches);
$item['id'] = self::URI . $matches[0];
preg_match('~(?<=href=\").*?(?=\")~', $cols[4], $matches);
$item['uri'] = $matches[0];
$item['timestamp'] = $timestamp;
preg_match('~(?<=href=\"/users/).*?(?=\")~', $cols[3], $matches);
$item['author'] = $matches[0];
preg_match('~(?<=/\">).*?(?=</a>)~', $cols[1], $matches);
$item['title'] = $matches[0];
preg_match('~(?<=green\">)\d+(?=</font>)~', $cols[8], $matches);
$item['seeders'] = $matches[0];
preg_match('~(?<=red\">)\d+(?=</font>)~', $cols[9], $matches);
$item['leechers'] = $matches[0];
preg_match('~(?<=>).*?(?=</td>)~', $cols[5], $matches);
$item['size'] = $matches[0];
$item['content'] = 'Uploaded by ' . $item['author']
. ' , Size ' . $item['size']
. '<br>seeders: '
. $item['seeders']
. ' | leechers: '
. $item['leechers']
. '<br><a href="'
. $item['id']
. '">info page</a>';
if(isset($item['title']))
$this->items[] = $item;
}
$cursorCount++;
}
$content = $table->find('tr', $cursorCount)->find('a', 1);
$cursorCount++;
$torrentInfo = $table->find('tr', $cursorCount);
$item['timestamp'] = $timestamp;
$item['title'] = $content->plaintext;
$item['id'] = self::URI . $content->href;
$item['uri'] = self::URI . $content->href;
$item['author'] = $torrentInfo->find('a[class=user]', 0)->plaintext;
$item['seeders'] = $torrentInfo->find('font[class=green]', 0)->plaintext;
$item['leechers'] = $torrentInfo->find('font[class=red]', 0)->plaintext;
$item['size'] = $torrentInfo->find('td', 3)->plaintext;
$item['content'] = 'Uploaded by ' . $item['author']
. ' , Size ' . $item['size']
. '<br>seeders: '
. $item['seeders']
. ' | leechers: '
. $item['leechers']
. '<br><a href="'
. $item['id']
. '">info page</a>';
$this->items[] = $item;
$cursorCount++;
}
}
}

112
bridges/DiscogsBridge.php Normal file
View File

@@ -0,0 +1,112 @@
<?php
class DiscogsBridge extends BridgeAbstract {
const MAINTAINER = 'teromene';
const NAME = 'DiscogsBridge';
const URI = 'https://www.discogs.com/';
const DESCRIPTION = 'Returns releases from discogs';
const PARAMETERS = array(
'Artist Releases' => array(
'artistid' => array(
'name' => 'Artist ID',
'type' => 'number',
)
),
'Label Releases' => array(
'labelid' => array(
'name' => 'Label ID',
'type' => 'number',
)
),
'User Wantlist' => array(
'username_wantlist' => array(
'name' => 'Username',
'type' => 'text',
)
),
'User Folder' => array(
'username_folder' => array(
'name' => 'Username',
'type' => 'text',
),
'folderid' => array(
'name' => 'Folder ID',
'type' => 'number',
)
)
);
public function collectData() {
if(!empty($this->getInput('artistid')) || !empty($this->getInput('labelid'))) {
if(!empty($this->getInput('artistid'))) {
$data = getContents('https://api.discogs.com/artists/'
. $this->getInput('artistid')
. '/releases?sort=year&sort_order=desc')
or returnServerError('Unable to query discogs !');
} elseif(!empty($this->getInput('labelid'))) {
$data = getContents('https://api.discogs.com/labels/'
. $this->getInput('labelid')
. '/releases?sort=year&sort_order=desc')
or returnServerError('Unable to query discogs !');
}
$jsonData = json_decode($data, true);
foreach($jsonData['releases'] as $release) {
$item = array();
$item['author'] = $release['artist'];
$item['title'] = $release['title'];
$item['id'] = $release['id'];
$resId = array_key_exists('main_release', $release) ? $release['main_release'] : $release['id'];
$item['uri'] = self::URI . $this->getInput('artistid') . '/release/' . $resId;
$item['timestamp'] = DateTime::createFromFormat('Y', $release['year'])->getTimestamp();
$item['content'] = $item['author'] . ' - ' . $item['title'];
$this->items[] = $item;
}
} elseif(!empty($this->getInput('username_wantlist')) || !empty($this->getInput('username_folder'))) {
if(!empty($this->getInput('username_wantlist'))) {
$data = getContents('https://api.discogs.com/users/'
. $this->getInput('username_wantlist')
. '/wants?sort=added&sort_order=desc')
or returnServerError('Unable to query discogs !');
$jsonData = json_decode($data, true)['wants'];
} elseif(!empty($this->getInput('username_folder'))) {
$data = getContents('https://api.discogs.com/users/'
. $this->getInput('username_folder')
. '/collection/folders/'
. $this->getInput('folderid')
.'/releases?sort=added&sort_order=desc')
or returnServerError('Unable to query discogs !');
$jsonData = json_decode($data, true)['releases'];
}
foreach($jsonData as $element) {
$infos = $element['basic_information'];
$item = array();
$item['title'] = $infos['title'];
$item['author'] = $infos['artists'][0]['name'];
$item['id'] = $infos['artists'][0]['id'];
$item['uri'] = self::URI . $infos['artists'][0]['id'] . '/release/' . $infos['id'];
$item['timestamp'] = strtotime($element['date_added']);
$item['content'] = $item['author'] . ' - ' . $item['title'];
$this->items[] = $item;
}
}
}
public function getURI() {
return self::URI;
}
public function getName() {
return static::NAME;
}
}

142
bridges/ETTVBridge.php Normal file
View File

@@ -0,0 +1,142 @@
<?php
class ETTVBridge extends BridgeAbstract {
const MAINTAINER = 'GregThib';
const NAME = 'ETTV';
const URI = 'https://www.ettv.tv/';
const DESCRIPTION = 'Returns list of 20 latest torrents for a specific search.';
const CACHE_TIMEOUT = 14400; // 4 hours
const PARAMETERS = array( array(
'query' => array(
'name' => 'Keywords',
'required' => true
),
'cat' => array(
'type' => 'list',
'name' => 'Category',
'values' => array(
'(ALL TYPES)' => '0',
'Anime: Movies' => '73',
'Anime: Dubbed/Subbed' => '74',
'Anime: Others' => '75',
'Books: Ebooks' => '53',
'Books: Magazines' => '54',
'Books: Comics' => '55',
'Books: Audio' => '56',
'Books: Others' => '68',
'Games: Windows' => '57',
'Games: Android' => '58',
'Games: Others' => '71',
'Movies: HD 1080p' => '1',
'Movies: HD 720p' => '2',
'Movies: UltraHD/4K' => '3',
'Movies: XviD' => '42',
'Movies: X264/H264' => '47',
'Movies: 3D' => '49',
'Movies: Dubs/Dual Audio' => '51',
'Movies: CAM/TS' => '65',
'Movies: BluRay Disc/Remux' => '66',
'Movies: DVDR' => '67',
'Movies: HEVC/x265' => '76',
'Music: MP3' => '59',
'Music: FLAC' => '60',
'Music: Music Videos' => '61',
'Music: Others' => '69',
'Software: Windows' => '62',
'Software: Android' => '63',
'Software: Mac' => '64',
'Software: Others' => '70',
'TV: HD/X264/H264' => '41',
'TV: SD/X264/H264' => '5',
'TV: TV Packs' => '7',
'TV: SD/XVID' => '50',
'TV: Sport' => '72',
'TV: HEVC/x265' => '77',
'Unsorted: Unsorted' => '78'
),
'defaultValue' => '(ALL TYPES)'
),
'status' => array(
'type' => 'list',
'name' => 'Status',
'values' => array(
'Active Transfers' => '0',
'Included Dead' => '1',
'Only Dead' => '2'
),
'defaultValue' => 'Included Dead'
),
'lang' => array(
'type' => 'list',
'name' => 'Lang',
'values' => array(
'(ALL)' => '0',
'Arabic' => '17',
'Chinese ' => '10',
'Danish' => '13',
'Dutch' => '11',
'English' => '1',
'Finnish' => '18',
'French' => '2',
'German' => '3',
'Greek' => '15',
'Hindi' => '8',
'Italian' => '4',
'Japanese' => '5',
'Korean' => '9',
'Polish' => '14',
'Russian' => '7',
'Spanish' => '6',
'Turkish' => '16'
),
'defaultValue' => '(ALL)'
)
));
public function collectData(){
// No control on inputs, because all have defaultValue set
$query_str = 'torrents-search.php';
$query_str .= '?search=' . urlencode('+'.str_replace(' ', ' +', $this->getInput('query')));
$query_str .= '&cat=' . $this->getInput('cat');
$query_str .= 'incldead&=' . $this->getInput('status');
$query_str .= '&lang=' . $this->getInput('lang');
$query_str .= '&sort=id&order=desc';
// Get results page
$html = getSimpleHTMLDOM(self::URI . $query_str)
or returnServerError('Could not request ' . $this->getName());
// Loop on each entry
foreach($html->find('table.table tr') as $element) {
if($element->parent->tag == 'thead') continue;
$entry = $element->find('td', 1)->find('a', 0);
// retrieve result page to get more details
$link = rtrim(self::URI, '/') . $entry->href;
$page = getSimpleHTMLDOM($link)
or returnServerError('Could not request page ' . $link);
// get details & download links
$details = $page->find('fieldset.download table', 0); // WHAT?? It should be the second one…
$dllinks = $page->find('div#downloadbox table', 0);
// fill item
$item = array();
$item['author'] = $details->children(6)->children(1)->plaintext;
$item['title'] = $entry->title;
$item['uri'] = $dllinks->children(0)->children(0)->children(0)->href;
$item['timestamp'] = strtotime($details->children(7)->children(1)->plaintext);
$item['content'] = '';
$item['content'] .= '<br/><b>Name: </b>' . $details->children(0)->children(1)->innertext;
$item['content'] .= '<br/><b>Lang: </b>' . $details->children(3)->children(1)->innertext;
$item['content'] .= '<br/><b>Size: </b>' . $details->children(4)->children(1)->innertext;
$item['content'] .= '<br/><b>Hash: </b>' . $details->children(5)->children(1)->innertext;
foreach($dllinks->children(0)->children(1)->find('a') as $dl) {
$item['content'] .= '<br/>' . $dl->outertext;
}
$item['content'] .= '<br/><br/>' . $details->children(1)->children(0)->innertext;
$this->items[] = $item;
}
}
}

View File

@@ -1,7 +1,7 @@
<?php
class EZTVBridge extends BridgeAbstract {
const MAINTAINER = "alexAubin";
const MAINTAINER = 'alexAubin';
const NAME = 'EZTV';
const URI = 'https://eztv.ch/';
const DESCRIPTION = 'Returns list of *recent* torrents for a specific show
@@ -23,15 +23,15 @@ on EZTV. Get showID from URLs in https://eztv.ch/shows/showID/show-full-name.';
$relativeDays = 0;
$relativeHours = 0;
foreach(explode(" ", $relativeReleaseTime) as $relativeTimeElement) {
if(substr($relativeTimeElement, -1) == "d") $relativeDays = substr($relativeTimeElement, 0, -1);
if(substr($relativeTimeElement, -1) == "h") $relativeHours = substr($relativeTimeElement, 0, -1);
foreach(explode(' ', $relativeReleaseTime) as $relativeTimeElement) {
if(substr($relativeTimeElement, -1) == 'd') $relativeDays = substr($relativeTimeElement, 0, -1);
if(substr($relativeTimeElement, -1) == 'h') $relativeHours = substr($relativeTimeElement, 0, -1);
}
return mktime(date('h') - $relativeHours, 0, 0, date('m'), date('d') - $relativeDays, date('Y'));
}
// Loop on show ids
$showList = explode(",", $this->getInput('i'));
$showList = explode(',', $this->getInput('i'));
foreach($showList as $showID) {
// Get show page

146
bridges/ElloBridge.php Normal file
View File

@@ -0,0 +1,146 @@
<?php
class ElloBridge extends BridgeAbstract {
const MAINTAINER = 'teromene';
const NAME = 'Ello Bridge';
const URI = 'https://ello.co/';
const CACHE_TIMEOUT = 4800; //2hours
const DESCRIPTION = 'Returns the newest posts for Ello';
const PARAMETERS = array(
'By User' => array(
'u' => array(
'name' => 'Username',
'required' => true,
'title' => 'Username'
)
),
'Search' => array(
's' => array(
'name' => 'Search',
'required' => true,
'title' => 'Search'
)
)
);
public function collectData() {
$header = array(
'Authorization: Bearer ' . $this->getAPIKey()
);
if(!empty($this->getInput('u'))) {
$postData = getContents(self::URI . 'api/v2/users/~' . urlencode($this->getInput('u')) . '/posts', $header) or
returnServerError('Unable to query Ello API.');
} else {
$postData = getContents(self::URI . 'api/v2/posts?terms=' . urlencode($this->getInput('s')), $header) or
returnServerError('Unable to query Ello API.');
}
$postData = json_decode($postData);
$count = 0;
foreach($postData->posts as $post) {
$item = array();
$item['author'] = $this->getUsername($post, $postData);
$item['timestamp'] = strtotime($post->created_at);
$item['title'] = $this->findText($post->summary);
$item['content'] = $this->getPostContent($post->body);
$item['enclosures'] = $this->getEnclosures($post, $postData);
$content = $post->body;
$this->items[] = $item;
$count += 1;
}
}
public function findText($path) {
foreach($path as $summaryElement) {
if($summaryElement->kind == 'text') {
return $summaryElement->data;
}
}
return '';
}
public function getPostContent($path) {
$content = '';
foreach($path as $summaryElement) {
if($summaryElement->kind == 'text') {
$content .= $summaryElement->data;
} elseif ($summaryElement->kind == 'image') {
$alt = '';
if(property_exists($summaryElement->data, 'alt')) {
$alt = $summaryElement->data->alt;
}
$content .= '<img src="' . $summaryElement->data->url . '" alt="' . $alt . '" />';
}
}
return $content;
}
public function getEnclosures($post, $postData) {
$assets = [];
foreach($post->links->assets as $asset) {
foreach($postData->linked->assets as $assetLink) {
if($asset == $assetLink->id) {
$assets[] = $assetLink->attachment->original->url;
break;
}
}
}
return $assets;
}
public function getUsername($post, $postData) {
foreach($postData->linked->users as $user) {
if($user->id == $post->links->author->id) {
return $user->username;
}
}
}
public function getAPIKey() {
$cache = Cache::create('FileCache');
$cache->setPath(CACHE_DIR);
$cache->setParameters(['key']);
$key = $cache->loadData();
if($key == null) {
$keyInfo = getContents(self::URI . 'api/webapp-token') or
returnServerError('Unable to get token.');
$key = json_decode($keyInfo)->token->access_token;
$cache->saveData($key);
}
return $key;
}
public function getName(){
if(!is_null($this->getInput('u'))) {
return $this->getInput('u') . ' - Ello Bridge';
}
return parent::getName();
}
}

View File

@@ -95,7 +95,7 @@ EOD;
. $pageID
. '&cursor={"card_id"%3A"videos"%2C"has_next_page"%3Atrue}&surface=mobile_page_home&unit_count=8';
$fileContent = file_get_contents($requestString);
$fileContent = getContents($requestString);
$articleIndex = 0;
$maxArticle = 3;
@@ -103,19 +103,19 @@ EOD;
$html = $this->buildContent($fileContent);
$author = $this->getInput('u');
foreach($html->find("article") as $content) {
foreach($html->find('article') as $content) {
$item = array();
$item['uri'] = "http://touch.facebook.com"
. $content->find("div[class='_52jc _5qc4 _24u0 _36xo']", 0)->find("a", 0)->getAttribute("href");
$item['uri'] = 'http://touch.facebook.com'
. $content->find("div[class='_52jc _5qc4 _24u0 _36xo']", 0)->find('a', 0)->getAttribute('href');
if($content->find("header", 0) !== null) {
$content->find("header", 0)->innertext = "";
if($content->find('header', 0) !== null) {
$content->find('header', 0)->innertext = '';
}
if($content->find("footer", 0) !== null) {
$content->find("footer", 0)->innertext = "";
if($content->find('footer', 0) !== null) {
$content->find('footer', 0)->innertext = '';
}
//Remove html nodes, keep only img, links, basic formatting
@@ -168,7 +168,7 @@ EOD;
$regex = implode(
'',
array(
"/timeline_unit",
'/timeline_unit',
"\\\\\\\\u00253A1",
"\\\\\\\\u00253A([0-9]*)",
"\\\\\\\\u00253A([0-9]*)",
@@ -182,29 +182,29 @@ EOD;
return implode(
'',
array(
"https://touch.facebook.com/pages_reaction_units/more/?page_id=",
'https://touch.facebook.com/pages_reaction_units/more/?page_id=',
$pageID,
"&cursor=%7B%22timeline_cursor%22%3A%22timeline_unit%3A1%3A",
'&cursor=%7B%22timeline_cursor%22%3A%22timeline_unit%3A1%3A',
$result[1],
"%3A",
'%3A',
$result[2],
"%3A",
'%3A',
$result[3],
"%3A",
'%3A',
$result[4],
"%22%2C%22timeline_section_cursor%22%3A%7B%7D%2C%22",
"has_next_page%22%3Atrue%7D&surface=mobile_page_home&unit_count=3"
'%22%2C%22timeline_section_cursor%22%3A%7B%7D%2C%22',
'has_next_page%22%3Atrue%7D&surface=mobile_page_home&unit_count=3'
)
);
}
//Builds the HTML from the encoded JS that Facebook provides.
private function buildContent($pageContent){
$regex = "/\\\"html\\\":\\\"(.*?)\\\",\\\"replace/";
// The html ends with:
// /div>","replaceifexists
$regex = '/\\"html\\":(\".+\/div>"),"replace/';
preg_match($regex, $pageContent, $result);
return str_get_html(html_entity_decode(json_decode('"' . $result[1] . '"')));
return str_get_html(html_entity_decode(json_decode($result[1])));
}
@@ -214,7 +214,7 @@ EOD;
$ctx = stream_context_create(array(
'http' => array(
'user_agent' => "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0",
'user_agent' => 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0',
'Accept' => 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
)
)
@@ -222,12 +222,12 @@ EOD;
$a = file_get_contents($pageURL, 0, $ctx);
//First request to get the cookie
$cookies = "";
$cookies = '';
foreach($http_response_header as $hdr) {
if(strpos($hdr, "Set-Cookie") !== false) {
$cLine = explode(":", $hdr)[1];
$cLine = explode(";", $cLine)[0];
$cookies .= ";" . $cLine;
if(strpos($hdr, 'Set-Cookie') !== false) {
$cLine = explode(':', $hdr)[1];
$cLine = explode(';', $cLine)[0];
$cookies .= ';' . $cLine;
}
}
@@ -239,7 +239,7 @@ EOD;
$context = stream_context_create(array(
'http' => array(
'user_agent' => "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0",
'user_agent' => 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0',
'header' => 'Cookie: ' . $cookies
)
)
@@ -247,12 +247,12 @@ EOD;
$pageContent = file_get_contents($page, 0, $context);
if(strpos($pageContent, "signup-button") != false) {
if(strpos($pageContent, 'signup-button') != false) {
return -1;
}
//Get the page ID if we don't have a captcha
$regex = "/page_id=([0-9]*)&/";
$regex = '/page_id=([0-9]*)&/';
preg_match($regex, $pageContent, $matches);
if(count($matches) > 0) {
@@ -260,7 +260,7 @@ EOD;
}
//Get the page ID if we do have a captcha
$regex = "/\"pageID\":\"([0-9]*)\"/";
$regex = '/"pageID":"([0-9]*)"/';
preg_match($regex, $pageContent, $matches);
return $matches[1];

54
bridges/FDroidBridge.php Normal file
View File

@@ -0,0 +1,54 @@
<?php
class FDroidBridge extends BridgeAbstract {
const MAINTAINER = 'Mitsukarenai';
const NAME = 'F-Droid Bridge';
const URI = 'https://f-droid.org/';
const CACHE_TIMEOUT = 60 * 60 * 2; // 2 hours
const DESCRIPTION = 'Returns latest added/updated apps on the open-source Android apps repository F-Droid';
const PARAMETERS = array( array(
'u' => array(
'name' => 'Widget selection',
'type' => 'list',
'required' => true,
'values' => array(
'Latest added apps' => 'added',
'Latest updated apps' => 'updated'
)
)
));
public function collectData(){
$url = self::URI;
$html = getSimpleHTMLDOM($url)
or returnServerError('Could not request F-Droid.');
// targetting the corresponding widget based on user selection
// "updated" is the 4th widget on the page, "added" is the 5th
switch($this->getInput('u')) {
case 'updated':
$html_widget = $html->find('div.sidebar-widget', 4);
break;
default:
$html_widget = $html->find('div.sidebar-widget', 5);
break;
}
// and now extracting app info from the selected widget (and yeah turns out icons are of heterogeneous sizes)
foreach($html_widget->find('a') as $element) {
$item = array();
$item['uri'] = self::URI . $element->href;
$item['title'] = $element->find('h4', 0)->plaintext;
$item['icon'] = $element->find('img', 0)->src;
$item['summary'] = $element->find('span.package-summary', 0)->plaintext;
$item['content'] = '
<a href="'.$item['uri'].'">
<img alt="" style="max-height:128px" src="'.$item['icon'].'">
</a><br>'.$item['summary'];
$this->items[] = $item;
}
}
}

View File

@@ -1,34 +1,257 @@
<?php
class FacebookBridge extends BridgeAbstract {
const MAINTAINER = 'teromene';
const MAINTAINER = 'teromene, logmanoriginal';
const NAME = 'Facebook';
const URI = 'https://www.facebook.com/';
const CACHE_TIMEOUT = 300; // 5min
const DESCRIPTION = 'Input a page title or a profile log. For a profile log,
please insert the parameter as follow : myExamplePage/132621766841117';
const PARAMETERS = array( array(
'u' => array(
'name' => 'Username',
'required' => true
),
'media_type' => array(
'name' => 'Media type',
'type' => 'list',
'required' => false,
'values' => array(
'All' => 'all',
'Video' => 'video',
'No Video' => 'novideo'
const PARAMETERS = array(
'User' => array(
'u' => array(
'name' => 'Username',
'required' => true
),
'defaultValue' => 'all'
'media_type' => array(
'name' => 'Media type',
'type' => 'list',
'required' => false,
'values' => array(
'All' => 'all',
'Video' => 'video',
'No Video' => 'novideo'
),
'defaultValue' => 'all'
),
'skip_reviews' => array(
'name' => 'Skip reviews',
'type' => 'checkbox',
'required' => false,
'defaultValue' => false,
'title' => 'Feed includes reviews when checked'
)
),
'Group' => array(
'g' => array(
'name' => 'Group',
'type' => 'text',
'required' => true,
'exampleValue' => 'https://www.facebook.com/groups/743149642484225',
'title' => 'Insert group name or facebook group URL'
)
)
));
);
private $authorName = '';
private $groupName = '';
public function collectData(){
public function getURI() {
$uri = self::URI;
switch($this->queriedContext) {
case 'Group':
$uri .= 'groups/' . $this->sanitizeGroup(filter_var($this->getInput('g'), FILTER_SANITIZE_URL));
break;
}
return $uri .= '?_fb_noscript=1';
}
public function collectData() {
switch($this->queriedContext) {
case 'Group':
$this->collectGroupData();
break;
case 'User':
$this->collectUserData();
break;
default:
returnClientError('Unknown context: "' . $this->queriedContext . '"!');
}
}
#region Group
private function collectGroupData() {
$header = array('Accept-Language: ' . getEnv('HTTP_ACCEPT_LANGUAGE') . "\r\n");
$html = getSimpleHTMLDOM($this->getURI(), $header)
or returnServerError('Failed loading facebook page: ' . $this->getURI());
if(!$this->isPublicGroup($html)) {
returnClientError('This group is not public! RSS-Bridge only supports public groups!');
}
defaultLinkTo($html, substr(self::URI, 0, strlen(self::URI) - 1));
$this->groupName = $this->extractGroupName($html);
$posts = $html->find('div.userContentWrapper')
or returnServerError('Failed finding posts!');
foreach($posts as $post) {
$item = array();
$item['uri'] = $this->extractGroupURI($post);
$item['title'] = $this->extractGroupTitle($post);
$item['author'] = $this->extractGroupAuthor($post);
$item['content'] = $this->extractGroupContent($post);
$item['timestamp'] = $this->extractGroupTimestamp($post);
$item['enclosures'] = $this->extractGroupEnclosures($post);
$this->items[] = $item;
}
}
private function sanitizeGroup($group) {
if(filter_var(
$group,
FILTER_VALIDATE_URL,
FILTER_FLAG_HOST_REQUIRED | FILTER_FLAG_PATH_REQUIRED)) {
// User provided a URL
$urlparts = parse_url($group);
if($urlparts['host'] !== parse_url(self::URI)['host']
&& 'www.' . $urlparts['host'] !== parse_url(self::URI)['host']) {
returnClientError('The host you provided is invalid! Received "'
. $urlparts['host']
. '", expected "'
. parse_url(self::URI)['host']
. '"!');
}
return explode('/', $urlparts['path'])[2];
} elseif(strpos($group, '/') !== false) {
returnClientError('The group you provided is invalid: ' . $group);
} else {
return $group;
}
}
private function isPublicGroup($html) {
// Facebook redirects to the groups about page for non-public groups
$about = $html->find('#pagelet_group_about', 0);
return !($about);
}
private function extractGroupName($html) {
$ogtitle = $html->find('meta[property="og:title"]', 0)
or returnServerError('Unable to find group title!');
return htmlspecialchars_decode($ogtitle->content, ENT_QUOTES);
}
private function extractGroupURI($post) {
$elements = $post->find('a')
or returnServerError('Unable to find URI!');
foreach($elements as $anchor) {
// Find the one that is a permalink
if(strpos($anchor->href, 'permalink') !== false) {
return $anchor->href;
}
}
return null;
}
private function extractGroupContent($post) {
$content = $post->find('div.userContent', 0)
or returnServerError('Unable to find user content!');
return $content->innertext . $content->next_sibling()->innertext;
}
private function extractGroupTimestamp($post) {
$element = $post->find('abbr[data-utime]', 0)
or returnServerError('Unable to find timestamp!');
return $element->getAttribute('data-utime');
}
private function extractGroupAuthor($post) {
$element = $post->find('img', 0)
or returnServerError('Unable to find author information!');
return $element->{'aria-label'};
}
private function extractGroupEnclosures($post) {
$elements = $post->find('div.userContent', 0)->next_sibling()->find('img');
$enclosures = array();
foreach($elements as $enclosure) {
$enclosures[] = $enclosure->src;
}
return empty($enclosures) ? null : $enclosures;
}
private function extractGroupTitle($post) {
$element = $post->find('h5', 0)
or returnServerError('Unable to find title!');
if(strpos($element->plaintext, 'shared') === false) {
$content = strip_tags($this->extractGroupContent($post));
return $this->extractGroupAuthor($post)
. ' posted: '
. substr(
$content,
0,
strpos(wordwrap($content, 64), "\n")
)
. '...';
}
return $element->plaintext;
}
#endregion
private function collectUserData(){
//Extract a string using start and end delimiters
function extractFromDelimiters($string, $start, $end){
@@ -95,7 +318,7 @@ class FacebookBridge extends BridgeAbstract {
if (isset($_SESSION['captcha_fields'], $_SESSION['captcha_action'])) {
$captcha_action = $_SESSION['captcha_action'];
$captcha_fields = $_SESSION['captcha_fields'];
$captcha_fields['captcha_response'] = preg_replace("/[^a-zA-Z0-9]+/", "", $_POST['captcha_response']);
$captcha_fields['captcha_response'] = preg_replace('/[^a-zA-Z0-9]+/', '', $_POST['captcha_response']);
$header = array("Content-type:
application/x-www-form-urlencoded\r\nReferer: $captcha_action\r\nCookie: noscript=1\r\n");
@@ -120,17 +343,44 @@ application/x-www-form-urlencoded\r\nReferer: $captcha_action\r\nCookie: noscrip
if(is_null($html)) {
$header = array('Accept-Language: ' . getEnv('HTTP_ACCEPT_LANGUAGE') . "\r\n");
// First character cannot be a forward slash
if(strpos($this->getInput('u'), "/") === 0) {
returnClientError('Remove leading slash "/" from the username!');
}
// Check if the user provided a fully qualified URL
if (filter_var($this->getInput('u'), FILTER_VALIDATE_URL)) {
$urlparts = parse_url($this->getInput('u'));
if($urlparts['host'] !== parse_url(self::URI)['host']) {
returnClientError('The host you provided is invalid! Received "'
. $urlparts['host']
. '", expected "'
. parse_url(self::URI)['host']
. '"!');
}
if(!array_key_exists('path', $urlparts)
|| $urlparts['path'] === '/') {
returnClientError('The URL you provided doesn\'t contain the user name!');
}
$user = explode('/', $urlparts['path'])[1];
$html = getSimpleHTMLDOM(self::URI . urlencode($user) . '?_fb_noscript=1', $header)
or returnServerError('No results for this query.');
if(!strpos($this->getInput('u'), "/")) {
$html = getSimpleHTMLDOM(self::URI . urlencode($this->getInput('u')) . '?_fb_noscript=1', $header)
or returnServerError('No results for this query.');
} else {
$html = getSimpleHTMLDOM(self::URI . 'pages/' . $this->getInput('u') . '?_fb_noscript=1', $header)
or returnServerError('No results for this query.');
// First character cannot be a forward slash
if(strpos($this->getInput('u'), '/') === 0) {
returnClientError('Remove leading slash "/" from the username!');
}
if(!strpos($this->getInput('u'), '/')) {
$html = getSimpleHTMLDOM(self::URI . urlencode($this->getInput('u')) . '?_fb_noscript=1', $header)
or returnServerError('No results for this query.');
} else {
$html = getSimpleHTMLDOM(self::URI . 'pages/' . $this->getInput('u') . '?_fb_noscript=1', $header)
or returnServerError('No results for this query.');
}
}
}
@@ -164,6 +414,12 @@ EOD;
}
//No captcha? We can carry on retrieving page contents :)
//First, we check wether the page is public or not
$loginForm = $html->find('._585r', 0);
if($loginForm != null) {
returnServerError('You must be logged in to view this page. This is not supported by RSS-Bridge.');
}
$element = $html
->find('#pagelet_timeline_main_column')[0]
->children(0)
@@ -189,6 +445,12 @@ EOD;
$posts = array($cell);
}
// Optionally skip reviews
if($this->getInput('skip_reviews')
&& !is_null($cell->find('#review_composer_container', 0))) {
continue;
}
foreach($posts as $post) {
// Check media type
switch($this->getInput('media_type')) {
@@ -259,7 +521,7 @@ EOD;
);
//Retrieve date of the post
$date = $post->find("abbr")[0];
$date = $post->find('abbr')[0];
if(isset($date) && $date->hasAttribute('data-utime')) {
$date = $date->getAttribute('data-utime');
} else {
@@ -290,9 +552,22 @@ EOD;
}
public function getName(){
if(!empty($this->authorName)) {
return isset($this->extraInfos['name']) ? $this->extraInfos['name'] : $this->authorName
. ' - Facebook Bridge';
switch($this->queriedContext) {
case 'User':
if(!empty($this->authorName)) {
return isset($this->extraInfos['name']) ? $this->extraInfos['name'] : $this->authorName
. ' - Facebook Bridge';
}
break;
case 'Group':
if(!empty($this->groupName)) {
return $this->groupName . ' - Facebook Bridge';
}
break;
}
return parent::getName();

View File

@@ -15,47 +15,47 @@ class FootitoBridge extends BridgeAbstract {
$content = trim($element->innertext);
$content = str_replace(
"<img",
'<img',
"<img style='float : left;'",
$content );
$content = str_replace(
"class=\"logo\"",
'class="logo"',
"style='float : left;'",
$content );
$content = str_replace(
"class=\"contenu\"",
'class="contenu"',
"style='margin-left : 60px;'",
$content );
$content = str_replace(
"class=\"responsive-comment\"",
'class="responsive-comment"',
"style='border-top : 1px #DDD solid; background-color : white; padding : 10px;'",
$content );
$content = str_replace(
"class=\"jaime\"",
'class="jaime"',
"style='display : none;'",
$content );
$content = str_replace(
"class=\"auteur-event responsive\"",
'class="auteur-event responsive"',
"style='display : none;'",
$content );
$content = str_replace(
"class=\"report-abuse-button\"",
'class="report-abuse-button"',
"style='display : none;'",
$content );
$content = str_replace(
"class=\"reaction clearfix\"",
'class="reaction clearfix"',
"style='margin : 10px 0px; padding : 5px; border-bottom : 1px #DDD solid;'",
$content );
$content = str_replace(
"class=\"infos\"",
'class="infos"',
"style='font-size : 0.7em;'",
$content );

View File

@@ -30,7 +30,7 @@ class FourchanBridge extends BridgeAbstract {
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError("Could not request 4chan, thread not found");
or returnServerError('Could not request 4chan, thread not found');
foreach($html->find('div.postContainer') as $element) {
$item = array();

View File

@@ -106,7 +106,7 @@ class GithubIssueBridge extends BridgeAbstract {
$content = $comment->parent()->innertext;
} else {
$title .= ' / ' . trim($comment->firstChild()->plaintext);
$content = "<pre>" . $comment->find('.comment-body', 0)->innertext . "</pre>";
$content = '<pre>' . $comment->find('.comment-body', 0)->innertext . '</pre>';
}
$item = array();

View File

@@ -3,7 +3,7 @@ class GoComicsBridge extends BridgeAbstract {
const MAINTAINER = 'sky';
const NAME = 'GoComics Unofficial RSS';
const URI = 'http://www.gocomics.com/';
const URI = 'https://www.gocomics.com/';
const CACHE_TIMEOUT = 21600; // 6h
const DESCRIPTION = 'The Unofficial GoComics RSS';
const PARAMETERS = array( array(
@@ -18,25 +18,27 @@ class GoComicsBridge extends BridgeAbstract {
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request GoComics: ' . $this->getURI());
foreach($html->find('div.comic__container') as $element) {
//Get info from first page
$author = preg_replace('/By /', '', $html->find('.media-subheading', 0)->plaintext);
$img = $element->find('.item-comic-image img', 0);
$link = $element->find('a.js-item-comic-link', 0);
$comic = $img->src;
$title = $link->title;
$url = $html->find('input.js-copy-link', 0)->value;
$date = substr($title, -10);
if (empty($title))
$title = 'GoComics ' . $this->getInput('comicname') . ' on ' . $date;
$date = strtotime($date);
$link = self::URI . $html->find('.gc-deck--cta-0', 0)->find('a', 0)->href;
for($i = 0; $i < 5; $i++) {
$item = array();
$item['id'] = $url;
$item['uri'] = $url;
$item['title'] = $title;
$item['author'] = preg_replace('/by /', '', $element->find('a.link-blended small', 0)->plaintext);
$item['timestamp'] = $date;
$item['content'] = '<img src="' . $comic . '" alt="' . $title . '" />';
$page = getSimpleHTMLDOM($link)
or returnServerError('Could not request GoComics: ' . $link);
$imagelink = $page->find('.img-fluid', 1)->src;
$date = explode('/', $link);
$item['id'] = $imagelink;
$item['uri'] = $link;
$item['author'] = $author;
$item['title'] = 'GoComics ' . $this->getInput('comicname');
$item['timestamp'] = DateTime::createFromFormat('Ymd', $date[5] . $date[6] . $date[7])->getTimestamp();
$item['content'] = '<img src="' . $imagelink . '" />';
$link = self::URI . $page->find('.js-previous-comic', 0)->href;
$this->items[] = $item;
}
}

View File

@@ -17,7 +17,7 @@ class GoogleSearchBridge extends BridgeAbstract {
const PARAMETERS = array(array(
'q' => array(
'name' => "keyword",
'name' => 'keyword',
'required' => true
)
));

View File

@@ -0,0 +1,61 @@
<?php
class GrandComicsDatabaseBridge extends BridgeAbstract {
const MAINTAINER = 'corenting';
const NAME = 'Grand Comics Database Bridge';
const URI = 'https://www.comics.org/';
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = 'Returns the latest comics added to a series timeline';
const PARAMETERS = array( array(
'series' => array(
'name' => 'Series id (from the timeline URL)',
'required' => true,
'exampleValue' => '63051',
),
));
public function collectData(){
$url = self::URI . 'series/' . $this->getInput('series') . '/details/timeline/';
$html = getSimpleHTMLDOM($url)
or returnServerError('Error while downloading the website content');
$table = $html->find('table', 0);
$list = array_reverse($table->find('[class^=row_even]'));
$seriesName = $html->find('span[id=series_name]', 0)->innertext;
// Get row headers
$rowHeaders = $table->find('th');
foreach($list as $article) {
// Skip empty rows
$emptyRow = $article->find('td.empty_month');
if (count($emptyRow) != 0) {
continue;
}
$rows = $article->find('td');
$key_date = $rows[0]->innertext;
// Get URL too
$uri = 'https://www.comics.org' . $article->find('a')[0]->href;
// Build content
$content = '';
for($i = 0; $i < count($rowHeaders); $i++) {
$headerItem = $rowHeaders[$i]->innertext;
$rowItem = $rows[$i]->innertext;
$content = $content . $headerItem . ': ' . $rowItem . '<br/>';
}
// Build final item
$item = array();
$item['title'] = $seriesName . ' - ' . $key_date;
$item['timestamp'] = strtotime($key_date);
$item['content'] = str_get_html($content);
$item['uri'] = $uri;
$this->items[] = $item;
}
}
}

1397
bridges/HotUKDealsBridge.php Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -11,60 +11,38 @@ class InstagramBridge extends BridgeAbstract {
'u' => array(
'name' => 'username',
'required' => true
),
'media_type' => array(
'name' => 'Media type',
'type' => 'list',
'required' => false,
'values' => array(
'Both' => 'all',
'Video' => 'video',
'Picture' => 'picture'
),
'defaultValue' => 'all'
)
),
array(
'h' => array(
'name' => 'hashtag',
'required' => true
),
)
),
'global' => array(
'media_type' => array(
'name' => 'Media type',
'type' => 'list',
'required' => false,
'values' => array(
'Both' => 'all',
'All' => 'all',
'Story' => 'story',
'Video' => 'video',
'Picture' => 'picture'
'Picture' => 'picture',
),
'defaultValue' => 'all'
)
)
);
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request Instagram.');
$innertext = null;
foreach($html->find('script') as $script) {
if('' === $script->innertext) {
continue;
}
$pos = strpos(trim($script->innertext), 'window._sharedData');
if(0 !== $pos) {
continue;
}
$innertext = $script->innertext;
break;
if(!is_null($this->getInput('h')) && $this->getInput('media_type') == 'story') {
returnClientError('Stories are not supported for hashtags!');
}
$json = trim(substr($innertext, $pos + 18), ' =;');
$data = json_decode($json);
$data = $this->getInstagramJSON($this->getURI());
if(!is_null($this->getInput('u'))) {
$userMedia = $data->entry_data->ProfilePage[0]->graphql->user->edge_owner_to_timeline_media->edges;
@@ -74,31 +52,83 @@ class InstagramBridge extends BridgeAbstract {
foreach($userMedia as $media) {
$media = $media->node;
// Check media type
switch($this->getInput('media_type')) {
case 'all': break;
case 'video':
if($media->is_video === false) continue 2;
break;
case 'picture':
if($media->is_video === true) continue 2;
break;
default: break;
if(!is_null($this->getInput('u'))) {
switch($this->getInput('media_type')) {
case 'all': break;
case 'video':
if($media->__typename != 'GraphVideo') continue 2;
break;
case 'picture':
if($media->__typename != 'GraphImage') continue 2;
break;
case 'story':
if($media->__typename != 'GraphSidecar') continue 2;
break;
default: break;
}
} else {
if($this->getInput('media_type') == 'video' && !$media->is_video) continue;
}
$item = array();
$item['uri'] = self::URI . 'p/' . $media->shortcode . '/';
$item['content'] = '<img src="' . htmlentities($media->display_url) . '" />';
if (isset($media->edge_media_to_caption->edges[0]->node->text)) {
$item['title'] = $media->edge_media_to_caption->edges[0]->node->text;
} else {
$item['title'] = basename($media->display_url);
}
if(!is_null($this->getInput('u')) && $media->__typename == 'GraphSidecar') {
$data = $this->getInstagramStory($item['uri']);
$item['content'] = $data[0];
$item['enclosures'] = $data[1];
} else {
$item['content'] = '<img src="' . htmlentities($media->display_url) . '" alt="'. $item['title'] . '" />';
$item['enclosures'] = array($media->display_url);
}
$item['timestamp'] = $media->taken_at_timestamp;
$this->items[] = $item;
}
}
protected function getInstagramStory($uri) {
$data = $this->getInstagramJSON($uri);
$mediaInfo = $data->entry_data->PostPage[0]->graphql->shortcode_media;
//Process the first element, that isn't in the node graph
$caption = $mediaInfo->edge_media_to_caption->edges[0]->node->text;
$enclosures = [$mediaInfo->display_url];
$content = '<img src="' . htmlentities($mediaInfo->display_url) . '" alt="'. $caption . '" />';
foreach($mediaInfo->edge_sidecar_to_children->edges as $media) {
$content .= '<img src="' . htmlentities($media->node->display_url) . '" alt="'. $caption . '" />';
$enclosures[] = $media->node->display_url;
}
return [$content, $enclosures];
}
protected function getInstagramJSON($uri) {
$html = getContents($uri)
or returnServerError('Could not request Instagram.');
$scriptRegex = '/window\._sharedData = (.*);<\/script>/';
preg_match($scriptRegex, $html, $matches, PREG_OFFSET_CAPTURE, 0);
return json_decode($matches[1][0]);
}
public function getName(){
if(!is_null($this->getInput('u'))) {
return $this->getInput('u') . ' - Instagram Bridge';

View File

@@ -1,465 +0,0 @@
<?php
class IsoHuntBridge extends BridgeAbstract {
const MAINTAINER = 'logmanoriginal';
const NAME = 'isoHunt Bridge';
const URI = 'https://isohunt.to/';
const CACHE_TIMEOUT = 300; //5min
const DESCRIPTION = 'Returns the latest results by category or search result';
const PARAMETERS = array(
/*
* Get feeds for one of the "latest" categories
* Notice: The categories "News" and "Top Searches" are received from the main page
* Elements are sorted by name ascending!
*/
'By "Latest" category' => array(
'latest_category' => array(
'name' => 'Latest category',
'type' => 'list',
'required' => true,
'title' => 'Select your category',
'defaultValue' => 'news',
'values' => array(
'Hot Torrents' => 'hot_torrents',
'News' => 'news',
'Releases' => 'releases',
'Torrents' => 'torrents'
)
)
),
/*
* Get feeds for one of the "torrent" categories
* Make sure to add new categories also to get_torrent_category_index($)!
* Elements are sorted by name ascending!
*/
'By "Torrent" category' => array(
'torrent_category' => array(
'name' => 'Torrent category',
'type' => 'list',
'required' => true,
'title' => 'Select your category',
'defaultValue' => 'anime',
'values' => array(
'Adult' => 'adult',
'Anime' => 'anime',
'Books' => 'books',
'Games' => 'games',
'Movies' => 'movies',
'Music' => 'music',
'Other' => 'other',
'Series & TV' => 'series_tv',
'Software' => 'software'
)
),
'torrent_popularity' => array(
'name' => 'Sort by popularity',
'type' => 'checkbox',
'title' => 'Activate to receive results by popularity'
)
),
/*
* Get feeds for a specific search request
*/
'Search torrent by name' => array(
'search_name' => array(
'name' => 'Name',
'required' => true,
'title' => 'Insert your search query',
'exampleValue' => 'Bridge'
),
'search_category' => array(
'name' => 'Category',
'type' => 'list',
'title' => 'Select your category',
'defaultValue' => 'all',
'values' => array(
'Adult' => 'adult',
'All' => 'all',
'Anime' => 'anime',
'Books' => 'books',
'Games' => 'games',
'Movies' => 'movies',
'Music' => 'music',
'Other' => 'other',
'Series & TV' => 'series_tv',
'Software' => 'software'
)
)
)
);
public function getURI(){
$uri = self::URI;
switch($this->queriedContext) {
case 'By "Latest" category':
switch($this->getInput('latest_category')) {
case 'hot_torrents':
$uri .= 'statistic/hot/torrents';
break;
case 'news':
break;
case 'releases':
$uri .= 'releases.php';
break;
case 'torrents':
$uri .= 'latest.php';
break;
}
break;
case 'By "Torrent" category':
$uri .= $this->buildCategoryUri(
$this->getInput('torrent_category'),
$this->getInput('torrent_popularity')
);
break;
case 'Search torrent by name':
$category = $this->getInput('search_category');
$uri .= $this->buildCategoryUri($category);
if($category !== 'movies')
$uri .= '&ihq=' . urlencode($this->getInput('search_name'));
break;
default: parent::getURI();
}
return $uri;
}
public function getName(){
switch($this->queriedContext) {
case 'By "Latest" category':
$categoryName = array_search(
$this->getInput('latest_category'),
self::PARAMETERS['By "Latest" category']['latest_category']['values']
);
$name = 'Latest ' . $categoryName . ' - ' . self::NAME;
break;
case 'By "Torrent" category':
$categoryName = array_search(
$this->getInput('torrent_category'),
self::PARAMETERS['By "Torrent" category']['torrent_category']['values']
);
$name = 'Category: ' . $categoryName . ' - ' . self::NAME;
break;
case 'Search torrent by name':
$categoryName = array_search(
$this->getInput('search_category'),
self::PARAMETERS['Search torrent by name']['search_category']['values']
);
$name = 'Search: "'
. $this->getInput('search_name')
. '" in category: '
. $categoryName . ' - '
. self::NAME;
break;
default: return parent::getName();
}
return $name;
}
public function collectData(){
$html = $this->loadHtml($this->getURI());
switch($this->queriedContext) {
case 'By "Latest" category':
switch($this->getInput('latest_category')) {
case 'hot_torrents':
$this->getLatestHotTorrents($html);
break;
case 'news':
$this->getLatestNews($html);
break;
case 'releases':
case 'torrents':
$this->getLatestTorrents($html);
break;
}
break;
case 'By "Torrent" category':
if($this->getInput('torrent_category') === 'movies') {
// This one is special (content wise)
$this->getMovieTorrents($html);
} else {
$this->getLatestTorrents($html);
}
break;
case 'Search torrent by name':
if($this->getInput('search_category') === 'movies') {
// This one is special (content wise)
$this->getMovieTorrents($html);
} else {
$this->getLatestTorrents($html);
}
break;
}
}
#region Helper functions for "Movie Torrents"
private function getMovieTorrents($html){
$container = $html->find('div#w0', 0);
if(!$container)
returnServerError('Unable to find torrent container!');
$torrents = $container->find('article');
if(!$torrents)
returnServerError('Unable to find torrents!');
foreach($torrents as $torrent) {
$anchor = $torrent->find('a', 0);
if(!$anchor)
returnServerError('Unable to find anchor!');
$date = $torrent->find('small', 0);
if(!$date)
returnServerError('Unable to find date!');
$item = array();
$item['uri'] = $this->fixRelativeUri($anchor->href);
$item['title'] = $anchor->title;
// $item['author'] =
$item['timestamp'] = strtotime($date->plaintext);
$item['content'] = $this->fixRelativeUri($torrent->innertext);
$this->items[] = $item;
}
}
#endregion
#region Helper functions for "Latest Hot Torrents"
private function getLatestHotTorrents($html){
$container = $html->find('div#serps', 0);
if(!$container)
returnServerError('Unable to find torrent container!');
$torrents = $container->find('tr');
if(!$torrents)
returnServerError('Unable to find torrents!');
// Remove first element (header row)
$torrents = array_slice($torrents, 1);
foreach($torrents as $torrent) {
$cell = $torrent->find('td', 0);
if(!$cell)
returnServerError('Unable to find cell!');
$element = $cell->find('a', 0);
if(!$element)
returnServerError('Unable to find element!');
$item = array();
$item['uri'] = $element->href;
$item['title'] = $element->plaintext;
// $item['author'] =
// $item['timestamp'] =
// $item['content'] =
$this->items[] = $item;
}
}
#endregion
#region Helper functions for "Latest News"
private function getLatestNews($html){
$container = $html->find('div#postcontainer', 0);
if(!$container)
returnServerError('Unable to find post container!');
$posts = $container->find('div.index-post');
if(!$posts)
returnServerError('Unable to find posts!');
foreach($posts as $post) {
$item = array();
$item['uri'] = $this->latestNewsExtractUri($post);
$item['title'] = $this->latestNewsExtractTitle($post);
$item['author'] = $this->latestNewsExtractAuthor($post);
$item['timestamp'] = $this->latestNewsExtractTimestamp($post);
$item['content'] = $this->latestNewsExtractContent($post);
$this->items[] = $item;
}
}
private function latestNewsExtractAuthor($post){
$author = $post->find('small', 0);
if(!$author)
returnServerError('Unable to find author!');
// The author is hidden within a string like: 'Posted by {author} on {date}'
preg_match('/Posted\sby\s(.*)\son/i', $author->innertext, $matches);
return $matches[1];
}
private function latestNewsExtractTimestamp($post){
$date = $post->find('small', 0);
if(!$date)
returnServerError('Unable to find date!');
// The date is hidden within a string like: 'Posted by {author} on {date}'
preg_match('/Posted\sby\s.*\son\s(.*)/i', $date->innertext, $matches);
$timestamp = strtotime($matches[1]);
// Make sure date is not in the future (dates are given like 'Nov. 20' without year)
if($timestamp > time()) {
$timestamp = strtotime('-1 year', $timestamp);
}
return $timestamp;
}
private function latestNewsExtractTitle($post){
$title = $post->find('a', 0);
if(!$title)
returnServerError('Unable to find title!');
return $title->plaintext;
}
private function latestNewsExtractUri($post){
$uri = $post->find('a', 0);
if(!$uri)
returnServerError('Unable to find uri!');
return $uri->href;
}
private function latestNewsExtractContent($post){
$content = $post->find('div', 0);
if(!$content)
returnServerError('Unable to find content!');
// Remove <h2>...</h2> (title)
foreach($content->find('h2') as $element) {
$element->outertext = '';
}
// Remove <small>...</small> (author)
foreach($content->find('small') as $element) {
$element->outertext = '';
}
return $content->innertext;
}
#endregion
#region Helper functions for "Latest Torrents", "Latest Releases" and "Torrent Category"
private function getLatestTorrents($html){
$container = $html->find('div#serps', 0);
if(!$container)
returnServerError('Unable to find torrent container!');
$torrents = $container->find('tr[data-key]');
if(!$torrents)
returnServerError('Unable to find torrents!');
foreach($torrents as $torrent) {
$item = array();
$item['uri'] = $this->latestTorrentsExtractUri($torrent);
$item['title'] = $this->latestTorrentsExtractTitle($torrent);
$item['author'] = $this->latestTorrentsExtractAuthor($torrent);
$item['timestamp'] = $this->latestTorrentsExtractTimestamp($torrent);
$item['content'] = ''; // There is no valuable content
$this->items[] = $item;
}
}
private function latestTorrentsExtractTitle($torrent){
$cell = $torrent->find('td.title-row', 0);
if(!$cell)
returnServerError('Unable to find title cell!');
$title = $cell->find('span', 0);
if(!$title)
returnServerError('Unable to find title!');
return $title->plaintext;
}
private function latestTorrentsExtractUri($torrent){
$cell = $torrent->find('td.title-row', 0);
if(!$cell)
returnServerError('Unable to find title cell!');
$uri = $cell->find('a', 0);
if(!$uri)
returnServerError('Unable to find uri!');
return $this->fixRelativeUri($uri->href);
}
private function latestTorrentsExtractAuthor($torrent){
$cell = $torrent->find('td.user-row', 0);
if(!$cell)
return; // No author
$user = $cell->find('a', 0);
if(!$user)
returnServerError('Unable to find user!');
return $user->plaintext;
}
private function latestTorrentsExtractTimestamp($torrent){
$cell = $torrent->find('td.date-row', 0);
if(!$cell)
returnServerError('Unable to find date cell!');
return strtotime('-' . $cell->plaintext, time());
}
#endregion
#region Generic helper functions
private function loadHtml($uri){
$html = getSimpleHTMLDOM($uri);
if(!$html)
returnServerError('Unable to load ' . $uri . '!');
return $html;
}
private function fixRelativeUri($uri){
return preg_replace('/\//i', self::URI, $uri, 1);
}
private function buildCategoryUri($category, $order_popularity = false){
switch($category) {
case 'anime': $index = 1; break;
case 'software' : $index = 2; break;
case 'games' : $index = 3; break;
case 'adult' : $index = 4; break;
case 'movies' : $index = 5; break;
case 'music' : $index = 6; break;
case 'other' : $index = 7; break;
case 'series_tv' : $index = 8; break;
case 'books': $index = 9; break;
case 'all':
default: $index = 0; break;
}
return 'torrents/?iht=' . $index . '&ihs=' . ($order_popularity ? 1 : 0) . '&age=0';
}
#endregion
}

353
bridges/JustETFBridge.php Normal file
View File

@@ -0,0 +1,353 @@
<?php
class JustETFBridge extends BridgeAbstract {
const NAME = 'justETF Bridge';
const URI = 'https://www.justetf.com';
const DESCRIPTION = 'Currently only supports the news feed';
const MAINTAINER = 'logmanoriginal';
const PARAMETERS = array(
'News' => array(
'full' => array(
'name' => 'Full Article',
'type' => 'checkbox',
'title' => 'Enable to load full articles'
)
),
'Profile' => array(
'isin' => array(
'name' => 'ISIN',
'type' => 'text',
'required' => true,
'pattern' => '[a-zA-Z]{2}[a-zA-Z0-9]{10}',
'title' => 'ISIN, consisting of 2-letter country code, 9-character identifier, check character'
),
'strategy' => array(
'name' => 'Include Strategy',
'type' => 'checkbox',
'defaultValue' => 'checked'
),
'description' => array(
'name' => 'Include Description',
'type' => 'checkbox',
'defaultValue' => 'checked'
)
),
'global' => array(
'lang' => array(
'name' => 'Language',
'required' => true,
'type' => 'list',
'values' => array(
'Englisch' => 'en',
'Deutsch' => 'de',
'Italiano' => 'it'
),
'defaultValue' => 'Englisch'
)
)
);
public function collectData() {
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Failed loading contents from ' . $this->getURI());
defaultLinkTo($html, static::URI);
switch($this->queriedContext) {
case 'News':
$this->collectNews($html);
break;
case 'Profile':
$this->collectProfile($html);
break;
}
}
public function getURI() {
$uri = static::URI;
if($this->getInput('lang')) {
$uri .= '/' . $this->getInput('lang');
}
switch($this->queriedContext) {
case 'News':
$uri .= '/news';
break;
case 'Profile':
$uri .= '/etf-profile.html?' . http_build_query(array(
'isin' => strtoupper($this->getInput('isin'))
));
break;
}
return $uri;
}
public function getName() {
$name = static::NAME;
$name .= ($this->queriedContext) ? ' - ' . $this->queriedContext : '';
switch($this->queriedContext) {
case 'News': break;
case 'Profile':
if($this->getInput('isin')) {
$name .= ' ISIN ' . strtoupper($this->getInput('isin'));
}
}
if($this->getInput('lang')) {
$name .= ' (' . strtoupper($this->getInput('lang')) . ')';
}
return $name;
}
#region Common
/**
* Fixes dates depending on the choosen language:
*
* de : dd.mm.yy
* en : dd.mm.yy
* it : dd/mm/yy
*
* Basically strtotime doesn't convert dates correctly due to formats
* being hard to interpret. So we use the DateTime object, manually
* fixing dates and times (set to 00:00:00.000).
*
* We don't know the timezone, so just assume +00:00 (or whatever
* DateTime chooses)
*/
private function fixDate($date) {
switch($this->getInput('lang')) {
case 'en':
case 'de':
$df = date_create_from_format('d.m.y', $date);
break;
case 'it':
$df = date_create_from_format('d/m/y', $date);
break;
}
date_time_set($df, 0, 0);
// debugMessage(date_format($df, 'U'));
return date_format($df, 'U');
}
private function extractImages($article) {
// Notice: We can have zero or more images (though it should mostly be 1)
$elements = $article->find('img');
$images = array();
foreach($elements as $img) {
// Skip the logo (mostly provided part of a hidden div)
if(substr($img->src, strrpos($img->src, '/') + 1) === 'logo.png')
continue;
$images[] = $img->src;
}
return $images;
}
#endregion
#region News
private function collectNews($html) {
$articles = $html->find('div.newsTopArticle')
or returnServerError('No articles found! Layout might have changed!');
foreach($articles as $article) {
$item = array();
// Common data
$item['uri'] = $this->extractNewsUri($article);
$item['timestamp'] = $this->extractNewsDate($article);
$item['title'] = $this->extractNewsTitle($article);
if($this->getInput('full')) {
$uri = $this->extractNewsUri($article);
$html = getSimpleHTMLDOMCached($uri)
or returnServerError('Failed loading full article from ' . $uri);
$fullArticle = $html->find('div.article', 0)
or returnServerError('No content found! Layout might have changed!');
defaultLinkTo($fullArticle, static::URI);
$item['author'] = $this->extractFullArticleAuthor($fullArticle);
$item['content'] = $this->extractFullArticleContent($fullArticle);
$item['enclosures'] = $this->extractImages($fullArticle);
} else {
$item['content'] = $this->extractNewsDescription($article);
$item['enclosures'] = $this->extractImages($article);
}
$this->items[] = $item;
}
}
private function extractNewsUri($article) {
$element = $article->find('a', 0)
or returnServerError('Anchor not found!');
return $element->href;
}
private function extractNewsDate($article) {
$element = $article->find('div.subheadline', 0)
or returnServerError('Date not found!');
// debugMessage($element->plaintext);
$date = trim(explode('|', $element->plaintext)[0]);
return $this->fixDate($date);
}
private function extractNewsDescription($article) {
$element = $article->find('span.newsText', 0)
or returnServerError('Description not found!');
$element->find('a', 0)->onclick = '';
// debugMessage($element->innertext);
return $element->innertext;
}
private function extractNewsTitle($article) {
$element = $article->find('h3', 0)
or returnServerError('Title not found!');
return $element->plaintext;
}
private function extractFullArticleContent($article) {
$element = $article->find('div.article_body', 0)
or returnServerError('Article body not found!');
// Remove teaser image
$element->find('img.teaser-img', 0)->outertext = '';
// Remove self advertisements
foreach($element->find('.call-action') as $adv) {
$adv->outertext = '';
}
// Remove tips
foreach($element->find('.panel-edu') as $tip) {
$tip->outertext = '';
}
// Remove inline scripts (used for i.e. interactive graphs) as they are
// rendered as a long series of strings
foreach($element->find('script') as $script) {
$script->outertext = '[Content removed! Visit site to see full contents!]';
}
return $element->innertext;
}
private function extractFullArticleAuthor($article) {
$element = $article->find('span[itemprop=name]', 0)
or returnServerError('Author not found!');
return $element->plaintext;
}
#endregion
#region Profile
private function collectProfile($html) {
$item = array();
$item['uri'] = $this->getURI();
$item['timestamp'] = $this->extractProfileDate($html);
$item['title'] = $this->extractProfiletitle($html);
$item['author'] = $this->extractProfileAuthor($html);
$item['content'] = $this->extractProfileContent($html);
$this->items[] = $item;
}
private function extractProfileDate($html) {
$element = $html->find('div.infobox div.vallabel', 0)
or returnServerError('Date not found!');
// debugMessage($element->plaintext);
$date = trim(explode("\r\n", $element->plaintext)[1]);
return $this->fixDate($date);
}
private function extractProfileTitle($html) {
$element = $html->find('span.h1', 0)
or returnServerError('Title not found!');
return $element->plaintext;
}
private function extractProfileContent($html) {
// There are a few thins we are interested:
// - Investment Strategy
// - Description
// - Quote
$strategy = $html->find('div.tab-container div.col-sm-6 p', 0)
or returnServerError('Investment Strategy not found!');
// Description requires a bit of cleanup due to lack of propper identification
$description = $html->find('div.headline', 5)
or returnServerError('Description container not found!');
$description = $description->parent();
foreach($description->find('div') as $div) {
$div->outertext = '';
}
$quote = $html->find('div.infobox div.val', 0)
or returnServerError('Quote not found!');
$quote_html = '<strong>Quote</strong><br><p>' . $quote . '</p>';
$strategy_html = '';
$description_html = '';
if($this->getInput('strategy') === true) {
$strategy_html = '<strong>Strategy</strong><br><p>' . $strategy . '</p><br>';
}
if($this->getInput('description') === true) {
$description_html = '<strong>Description</strong><br><p>' . $description . '</p><br>';
}
return $strategy_html . $description_html . $quote_html;
}
private function extractProfileAuthor($html) {
// Use ISIN + WKN as author
// Notice: "identfier" is not a typo [sic]!
$element = $html->find('span.identfier', 0)
or returnServerError('Author not found!');
return $element->plaintext;
}
#endregion
}

View File

@@ -58,7 +58,7 @@ class KununuBridge extends BridgeAbstract {
break;
}
return self::URI . $site . '/' . $company . '/' . $section;
return self::URI . $site . '/' . $company . '/' . $section . '?sort=update_time_desc';
}
return parent::getURI();
@@ -135,8 +135,8 @@ class KununuBridge extends BridgeAbstract {
* Encodes unmlauts in the given text
*/
private function encodeUmlauts($text){
$umlauts = Array("/ä/","/ö/","/ü/","/Ä/","/Ö/","/Ü/","/ß/");
$replace = Array("ae","oe","ue","Ae","Oe","Ue","ss");
$umlauts = Array('/ä/','/ö/','/ü/','/Ä/','/Ö/','/Ü/','/ß/');
$replace = Array('ae','oe','ue','Ae','Oe','Ue','ss');
return preg_replace($umlauts, $replace, $text);
}
@@ -190,11 +190,11 @@ class KununuBridge extends BridgeAbstract {
* Returns the URI from a given article
*/
private function extractArticleUri($article){
$anchor = $article->find('ku-company-review-more', 0);
$anchor = $article->find('h1.review-title a', 0);
if(is_null($anchor))
returnServerError('Cannot find article URI!');
return self::URI . $anchor->{'review-url'};
return self::URI . $anchor->href;
}
/**

View File

@@ -1,11 +1,10 @@
<?php
class LeBonCoinBridge extends BridgeAbstract {
const MAINTAINER = '16mhz';
const MAINTAINER = 'jacknumber';
const NAME = 'LeBonCoin';
const URI = 'https://www.leboncoin.fr/';
const DESCRIPTION = 'Returns most recent results from LeBonCoin for a
region, and optionally a category and a keyword .';
const DESCRIPTION = 'Returns most recent results from LeBonCoin';
const PARAMETERS = array(
array(
@@ -14,125 +13,137 @@ region, and optionally a category and a keyword .';
'name' => 'Région',
'type' => 'list',
'values' => array(
'Toute la France' => 'ile_de_france/occasions',
'Alsace' => 'alsace',
'Aquitaine' => 'aquitaine',
'Auvergne' => 'auvergne',
'Basse Normandie' => 'basse_normandie',
'Bourgogne' => 'bourgogne',
'Bretagne' => 'bretagne',
'Centre' => 'centre',
'Champagne Ardenne' => 'champagne_ardenne',
'Corse' => 'corse',
'Franche Comté' => 'franche_comte',
'Haute Normandie' => 'haute_normandie',
'Ile de France' => 'ile_de_france',
'Languedoc Roussillon' => 'languedoc_roussillon',
'Limousin' => 'limousin',
'Lorraine' => 'lorraine',
'Midi Pyrénées' => 'midi_pyrenees',
'Nord Pas De Calais' => 'nord_pas_de_calais',
'Pays de la Loire' => 'pays_de_la_loire',
'Picardie' => 'picardie',
'Poitou Charentes' => 'poitou_charentes',
'Provence Alpes Côte d\'Azur' => 'provence_alpes_cote_d_azur',
'Rhône-Alpes' => 'rhone_alpes',
'Guadeloupe' => 'guadeloupe',
'Martinique' => 'martinique',
'Guyane' => 'guyane',
'Réunion' => 'reunion'
'Toute la France' => '',
'Alsace' => '1',
'Aquitaine' => '2',
'Auvergne' => '3',
'Basse Normandie' => '4',
'Bourgogne' => '5',
'Bretagne' => '6',
'Centre' => '7',
'Champagne Ardenne' => '8',
'Corse' => '9',
'Franche Comté' => '10',
'Haute Normandie' => '11',
'Ile de France' => '12',
'Languedoc Roussillon' => '13',
'Limousin' => '14',
'Lorraine' => '15',
'Midi Pyrénées' => '16',
'Nord Pas De Calais' => '17',
'Pays de la Loire' => '18',
'Picardie' => '19',
'Poitou Charentes' => '20',
'Provence Alpes Côte d\'Azur' => '21',
'Rhône-Alpes' => '22',
'Guadeloupe' => '23',
'Martinique' => '24',
'Guyane' => '25',
'Réunion' => '26'
)
),
'c' => array(
'name' => 'Catégorie',
'type' => 'list',
'values' => array(
'TOUS' => '',
'EMPLOI' => '_emploi_',
'Toutes catégories' => '',
'EMPLOI' => array(
'Emploi et recrutement' => '71',
'Offres d\'emploi et jobs' => '33'
),
'VEHICULES' => array(
'Tous' => '_vehicules_',
'Voitures' => 'voitures',
'Motos' => 'motos',
'Caravaning' => 'caravaning',
'Utilitaires' => 'utilitaires',
'Équipement Auto' => 'equipement_auto',
'Équipement Moto' => 'equipement_moto',
'Équipement Caravaning' => 'equipement_caravaning',
'Nautisme' => 'nautisme',
'Équipement Nautisme' => 'equipement_nautisme'
'Tous' => '1',
'Voitures' => '2',
'Motos' => '3',
'Caravaning' => '4',
'Utilitaires' => '5',
'Equipement Auto' => '6',
'Equipement Moto' => '44',
'Equipement Caravaning' => '50',
'Nautisme' => '7',
'Equipement Nautisme' => '51'
),
'IMMOBILIER' => array(
'Tous' => '_immobilier_',
'Ventes immobilières' => 'ventes_immobilieres',
'Locations' => 'locations',
'Colocations' => 'colocations',
'Bureaux & Commerces' => 'bureaux_commerces'
'Tous' => '8',
'Ventes immobilières' => '9',
'Locations' => '10',
'Colocations' => '11',
'Bureaux & Commerces' => '13'
),
'VACANCES' => array(
'Tous' => '_vacances_',
'Location gîtes' => 'locations_gites',
'Chambres d\'hôtes' => 'chambres_d_hotes',
'Campings' => 'campings',
'Hôtels' => 'hotels',
'Hébergements insolites' => 'hebergements_insolites'
'Tous' => '66',
'Locations & Gîtes' => '12',
'Chambres d\'hôtes' => '67',
'Campings' => '68',
'Hôtels' => '69',
'Hébergements insolites' => '70'
),
'MULTIMEDIA' => array(
'Tous' => '_multimedia_',
'Informatique' => 'informatique',
'Consoles & Jeux vidéo' => 'consoles_jeux_video',
'Image & Son' => 'image_son',
'Téléphonie' => 'telephonie'
'Tous' => '14',
'Informatique' => '15',
'Consoles & Jeux vidéo' => '43',
'Image & Son' => '16',
'Téléphonie' => '17'
),
'LOISIRS' => array(
'Tous' => '_loisirs_',
'DVD / Films' => 'dvd_films',
'CD / Musique' => 'cd_musique',
'Livres' => 'livres',
'Animaux' => 'animaux',
'Vélos' => 'velos',
'Sports & Hobbies' => 'sports_hobbies',
'Instruments de musique' => 'instruments_de_musique',
'Collection' => 'collection',
'Jeux & Jouets' => 'jeux_jouets',
'Vins & Gastronomie' => 'vins_gastronomie'
'Tous' => '24',
'DVD / Films' => '25',
'CD / Musique' => '26',
'Livres' => '27',
'Animaux' => '28',
'Vélos' => '55',
'Sports & Hobbies' => '29',
'Instruments de musique' => '30',
'Collection' => '40',
'Jeux & Jouets' => '41',
'Vins & Gastronomie' => '48'
),
'MATÉRIEL PROFESSIONNEL' => array(
'Tous' => '_materiel_professionnel_',
'Matériel Agricole' => 'mateiel_agricole',
'Transport - Manutention' => 'transport_manutention',
'BTP - Chantier - Gros-œuvre' => 'btp_chantier_gros_oeuvre',
'Outillage - Matériaux 2nd-œuvre' => 'outillage_materiaux_2nd_oeuvre',
'Équipements Industriels' => 'equipement_industriels',
'Restauration - Hôtellerie' => 'restauration_hotellerie',
'Fournitures de Bureau' => 'fournitures_de_bureau',
'Commerces & Marchés' => 'commerces_marches',
'Matériel médical' => 'materiel_medical'
'MATERIEL PROFESSIONNEL' => array(
'Tous' => '56',
'Matériel Agricole' => '57',
'Transport - Manutention' => '58',
'BTP - Chantier Gros-oeuvre' => '59',
'Outillage - Matériaux 2nd-oeuvre' => '60',
'Équipements Industriels' => '32',
'Restauration - Hôtellerie' => '61',
'Fournitures de Bureau' => '62',
'Commerces & Marchés' => '63',
'Matériel Médical' => '64'
),
'SERVICES' => array(
'Tous' => '_services_',
'Prestations de services' => 'prestations_de_services',
'Billetterie' => 'billetterie',
'Évènements' => 'evenements',
'Cours particuliers' => 'cours_particuliers',
'Covoiturage' => 'covoiturage'
'Tous' => '31',
'Prestations de services' => '34',
'Billetterie' => '35',
'Evénements' => '49',
'Cours particuliers' => '36',
'Covoiturage' => '65'
),
'MAISON' => array(
'Tous' => '_maison_',
'Ameublement' => 'ameublement',
'Électroménager' => 'electromenager',
'Arts de la table' => 'arts_de_la_table',
'Décoration' => 'decoration',
'Linge de maison' => 'linge_de_maison',
'Bricolage' => 'bricolage',
'Jardinage' => 'jardinage',
'Vêtements' => 'vetements',
'Chaussures' => 'chaussures',
'Accessoires & Bagagerie' => 'accessoires_bagagerie',
'Montres & Bijoux' => 'montres_bijoux',
'Équipement bébé' => 'equipement_bebe',
'Vêtements bébé' => 'vetements_bebe'
'Tous' => '18',
'Ameublement' => '19',
'Electroménager' => '20',
'Arts de la table' => '45',
'Décoration' => '39',
'Linge de maison' => '46',
'Bricolage' => '21',
'Jardinage' => '52',
'Vêtements' => '22',
'Chaussures' => '53',
'Accessoires & Bagagerie' => '47',
'Montres & Bijoux' => '42',
'Equipement bébé' => '23',
'Vêtements bébé' => '54',
),
'AUTRES' => 'autres'
'AUTRES' => '37'
)
),
'o' => array(
'name' => 'Vendeur',
'type' => 'list',
'values' => array(
'Tous' => '',
'Particuliers' => 'private',
'Professionnels' => 'pro',
)
)
)
@@ -140,50 +151,71 @@ region, and optionally a category and a keyword .';
public function collectData(){
$category = $this->getInput('c');
if(empty($category)) {
$category = 'annonces';
$params = array(
'text' => $this->getInput('k'),
'region' => $this->getInput('r'),
'category' => $this->getInput('c'),
'owner_type' => $this->getInput('o'),
);
$url = self::URI . 'recherche/?' . http_build_query($params);
$html = getContents($url)
or returnServerError('Could not request LeBonCoin. Tried: ' . $url);
if(!preg_match('/^<script>window.FLUX_STATE[^\r\n]*/m', $html, $matches)) {
returnServerError('Could not parse JSON in page content.');
}
$html = getSimpleHTMLDOM(self::URI
. $category
. '/offres/'
. $this->getInput('r')
. '/?f=a&th=1&q='
. urlencode($this->getInput('k')))
or returnServerError('Could not request LeBonCoin.');
$clean_match = str_replace(
array('</script>', '<script>window.FLUX_STATE = '),
array('', ''),
$matches[0]
);
$json = json_decode($clean_match);
$list = $html->find('.tabsContent', 0);
if($list === null) {
if($json->adSearch->data->total === 0) {
return;
}
$tags = $list->find('li');
foreach($json->adSearch->data->ads as $element) {
foreach($tags as $element) {
$item['title'] = $element->subject;
$item['content'] = $element->body;
$item['date'] = $element->index_date;
$item['timestamp'] = strtotime($element->index_date);
$item['uri'] = $element->url;
$item['ad_type'] = $element->ad_type;
$item['author'] = $element->owner->name;
$element = $element->find('a', 0);
if(isset($element->location->city)) {
$item = array();
$item['uri'] = $element->href;
$title = html_entity_decode($element->getAttribute('title'));
$content_image = $element->find('div.item_image', 0)->find('.lazyload', 0);
$item['city'] = $element->location->city;
$item['content'] .= ' -- ' . $element->location->city;
if($content_image !== null) {
$content = '<img src="' . $content_image->getAttribute('data-imgsrc') . '" alt="thumbnail">';
} else {
$content = "";
}
$date = $element->find('aside.item_absolute', 0)->find('p.item_sup', 0);
$detailsList = $element->find('section.item_infos', 0);
if(isset($element->location->zipcode)) {
$item['zipcode'] = $element->location->zipcode;
}
for($i = 0; $i <= 1; $i++) $content .= $detailsList->find('p.item_supp', $i)->plaintext;
$price = $detailsList->find('h3.item_price', 0);
$content .= $price === null ? '' : $price->plaintext;
if(isset($element->price)) {
$item['price'] = $element->price[0];
$item['content'] .= ' -- ' . current($element->price) . '€';
}
if(isset($element->images->urls)) {
$item['thumbnail'] = $element->images->thumb_url;
$item['enclosures'] = array();
foreach($element->images->urls as $image) {
$item['enclosures'][] = $image;
}
}
$item['title'] = $title;
$item['content'] = $content . $date;
$this->items[] = $item;
}
}

View File

@@ -22,16 +22,16 @@ class LesJoiesDuCodeBridge extends BridgeAbstract {
// retrieve .gif instead of static .jpg
$images = $temp->find('p img');
foreach($images as $image) {
$img_src = str_replace(".jpg", ".gif", $image->src);
$img_src = str_replace('.jpg', '.gif', $image->src);
$image->src = $img_src;
}
$content = $temp->innertext;
$auteur = $temp->find('i', 0);
$pos = strpos($auteur->innertext, "by");
$pos = strpos($auteur->innertext, 'by');
if($pos > 0) {
$auteur = trim(str_replace("*/", "", substr($auteur->innertext, ($pos + 2))));
$auteur = trim(str_replace('*/', '', substr($auteur->innertext, ($pos + 2))));
$item['author'] = $auteur;
}

View File

@@ -100,7 +100,7 @@ class MangareaderBridge extends BridgeAbstract {
case 'Get popular mangas':
// Find manga name within "Popular mangas for ..."
$pagetitle = $xpath->query(".//*[@id='bodyalt']/h1")->item(0)->nodeValue;
$this->request = substr($pagetitle, 0, strrpos($pagetitle, " -"));
$this->request = substr($pagetitle, 0, strrpos($pagetitle, ' -'));
$this->getPopularMangas($xpath);
break;
case 'Get manga updates':
@@ -120,7 +120,7 @@ class MangareaderBridge extends BridgeAbstract {
// Return some dummy-data if no content available
if(empty($this->items)) {
$item = array();
$item['content'] = "<p>No updates available</p>";
$item['content'] = '<p>No updates available</p>';
$this->items[] = $item;
}
@@ -143,18 +143,18 @@ class MangareaderBridge extends BridgeAbstract {
$item['title'] = htmlspecialchars($manga->nodeValue);
// Add each chapter to the feed
$item['content'] = "";
$item['content'] = '';
foreach ($chapters as $chapter) {
if($item['content'] <> "") {
$item['content'] .= "<br>";
if($item['content'] <> '') {
$item['content'] .= '<br>';
}
$item['content'] .= "<a href='"
. self::URI
. htmlspecialchars($chapter->getAttribute('href'))
. "'>"
. htmlspecialchars($chapter->nodeValue)
. "</a>";
. '</a>';
}
$this->items[] = $item;
@@ -211,13 +211,13 @@ EOD;
foreach ($chapters as $chapter) {
$item = array();
$item['title'] = htmlspecialchars($xpath->query("td[1]", $chapter)
$item['title'] = htmlspecialchars($xpath->query('td[1]', $chapter)
->item(0)
->nodeValue);
$item['uri'] = self::URI . $xpath->query("td[1]/a", $chapter)
$item['uri'] = self::URI . $xpath->query('td[1]/a', $chapter)
->item(0)
->getAttribute('href');
$item['timestamp'] = strtotime($xpath->query("td[2]", $chapter)
$item['timestamp'] = strtotime($xpath->query('td[2]', $chapter)
->item(0)
->nodeValue);
array_unshift($this->items, $item);
@@ -227,12 +227,12 @@ EOD;
public function getURI(){
switch($this->queriedContext) {
case 'Get latest updates':
$path = "latest";
$path = 'latest';
break;
case 'Get popular mangas':
$path = "popular";
if($this->getInput('category') !== "all") {
$path .= "/" . $this->getInput('category');
$path = 'popular';
if($this->getInput('category') !== 'all') {
$path .= '/' . $this->getInput('category');
}
break;
case 'Get manga updates':

144
bridges/MydealsBridge.php Normal file
View File

@@ -0,0 +1,144 @@
<?php
require_once(__DIR__ . '/DealabsBridge.php');
class MydealsBridge extends PepperBridgeAbstract {
const NAME = 'Mydeals bridge';
const URI = 'https://www.mydealz.de/';
const DESCRIPTION = 'Zeigt die Deals von mydeals.de';
const MAINTAINER = 'sysadminstory';
const PARAMETERS = array(
'Suche nach Stichworten' => array (
'q' => array(
'name' => 'Stichworten',
'type' => 'text',
'required' => true
),
'hide_expired' => array(
'name' => 'Abgelaufenes ausblenden',
'type' => 'checkbox',
'required' => 'true'
),
'hide_local' => array(
'name' => 'Lokales ausblenden',
'type' => 'checkbox',
'title' => 'Deals im physischen Geschäft ausblenden',
'required' => 'true'
),
'priceFrom' => array(
'name' => 'Minimaler Preis',
'type' => 'text',
'title' => 'Minmaler Preis in Euros',
'required' => 'false',
'defaultValue' => ''
),
'priceTo' => array(
'name' => 'Maximaler Preis',
'type' => 'text',
'title' => 'maximaler Preis in Euro',
'required' => 'false',
'defaultValue' => ''
),
),
'Deals pro Gruppen' => array(
'group' => array(
'name' => 'Gruppen',
'type' => 'list',
'required' => 'true',
'title' => 'Gruppe, deren Deals angezeigt werden müssen',
'values' => array(
'Elektronik' => 'elektronik',
'Handy & Smartphone' => 'smartphone',
'Gaming' => 'gaming',
'Software' => 'apps-software',
'Fashion Frauen' => 'fashion-frauen',
'Fashion Männer' => 'fashion-accessoires',
'Beauty & Gesundheit' => 'beauty',
'Family & Kids' => 'family-kids',
'Essen & Trinken' => 'food',
'Freizeit & Reisen' => 'reisen',
'Haushalt & Garten' => 'home-living',
'Entertainment' => 'entertainment',
'Verträge & Finanzen' => 'vertraege-finanzen',
'Coupons' => 'coupons',
)
),
'order' => array(
'name' => 'sortieren nach',
'type' => 'list',
'required' => 'true',
'title' => 'Sortierung der deals',
'values' => array(
'Vom heißesten zum kältesten Deal' => '',
'Vom jüngsten Deal zum ältesten' => '-new',
'Vom am meisten kommentierten Deal zum am wenigsten kommentierten Deal' => '-discussed'
)
)
)
);
public $lang = array(
'bridge-uri' => SELF::URI,
'bridge-name' => SELF::NAME,
'context-keyword' => 'Suche nach Stichworten',
'context-group' => 'Deals pro Gruppen',
'uri-group' => '/gruppe/',
'request-error' => 'Could not request mydeals',
'no-results' => 'Ups, wir konnten keine Deals zu',
'relative-date-indicator' => array(
'vor',
'seit'
),
'price' => 'Preis',
'shipping' => 'Versand',
'origin' => 'Ursprung',
'discount' => 'Rabatte',
'title-keyword' => 'Suche',
'title-group' => 'Gruppe',
'local-months' => array(
'Jan',
'Feb',
'Mär',
'Apr',
'Mai',
'Jun',
'Jul',
'Aug',
'Sep',
'Okt',
'Nov',
'Dez',
'.'
),
'local-time-relative' => array(
'eingestellt vor ',
'm',
'h,',
'day',
'days',
'month',
'year',
'and '
),
'date-prefixes' => array(
'eingestellt am ',
'lokal ',
'aktualisiert ',
),
'relative-date-alt-prefixes' => array(
'aktualisiert vor ',
'kommentiert vor ',
'heiß seit '
),
'relative-date-ignore-suffix' => array(
'/von.*$/'
),
'localdeal' => array(
'Lokal ',
'Läuft bis '
)
);
}

View File

@@ -12,7 +12,7 @@ class NasaApodBridge extends BridgeAbstract {
$html = getSimpleHTMLDOM(self::URI . 'archivepix.html')
or returnServerError('Error while downloading the website content');
$list = explode("<br>", $html->find('b', 0)->innertext);
$list = explode('<br>', $html->find('b', 0)->innertext);
for($i = 0; $i < 3; $i++) {
$line = $list[$i];
@@ -32,7 +32,7 @@ class NasaApodBridge extends BridgeAbstract {
$explanation = $picture_html->find('p', 2)->innertext;
//Extract date from the picture page
$date = explode(" ", $picture_html->find('p', 1)->innertext);
$date = explode(' ', $picture_html->find('p', 1)->innertext);
$item['timestamp'] = strtotime($date[4] . $date[3] . $date[2]);
//Other informations

View File

@@ -0,0 +1,57 @@
<?php
class NotAlwaysBridge extends BridgeAbstract {
const MAINTAINER = 'mozes';
const NAME = 'Not Always family Bridge';
const URI = 'https://notalwaysright.com/';
const DESCRIPTION = 'Returns the latest stories';
const CACHE_TIMEOUT = 1800; // 30 minutes
const PARAMETERS = array( array(
'filter' => array(
'type' => 'list',
'name' => 'Filter',
'values' => array(
'All' => 'all',
'Right' => 'right',
'Working' => 'working',
'Romantic' => 'romantic',
'Related' => 'related',
'Learning' => 'learning',
'Friendly' => 'friendly',
'Hopeless' => 'hopeless',
'Unfiltered' => 'unfiltered'
),
'required' => true
)
));
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request NotAlways.');
foreach($html->find('.post') as $post) {
#print_r($post);
$item = array();
$item['uri'] = $post->find('h1', 0)->find('a', 0)->href;
$item['content'] = $post;
$item['title'] = $post->find('h1', 0)->find('a', 0)->innertext;
$this->items[] = $item;
}
}
public function getName(){
if(!is_null($this->getInput('filter'))) {
return $this->getInput('filter') . ' - NotAlways Bridge';
}
return parent::getName();
}
public function getURI(){
if(!is_null($this->getInput('filter'))) {
return self::URI . $this->getInput('filter') . '/';
}
return parent::getURI();
}
}

View File

@@ -44,7 +44,7 @@ class PinterestBridge extends FeedExpander {
$pattern = '/https\:\/\/i\.pinimg\.com\/[a-zA-Z0-9]*x\//';
foreach($this->items as $item) {
$item["content"] = preg_replace($pattern, 'https://i.pinimg.com/originals/', $item["content"]);
$item['content'] = preg_replace($pattern, 'https://i.pinimg.com/originals/', $item['content']);
$newitems[] = $item;
}
$this->items = $newitems;
@@ -64,10 +64,10 @@ class PinterestBridge extends FeedExpander {
// provide even less info. Thus we attempt multiple options.
$item['title'] = trim($result['title']);
if($item['title'] === "")
if($item['title'] === '')
$item['title'] = trim($result['rich_summary']['display_name']);
if($item['title'] === "")
if($item['title'] === '')
$item['title'] = trim($result['grid_description']);
$item['timestamp'] = strtotime($result['created_at']);

73
bridges/PixivBridge.php Normal file
View File

@@ -0,0 +1,73 @@
<?php
class PixivBridge extends BridgeAbstract {
const MAINTAINER = 'teromene';
const NAME = 'Pixiv Bridge';
const URI = 'https://www.pixiv.net/';
const DESCRIPTION = 'Returns the tag search from pixiv.net';
const PARAMETERS = array( array(
'tag' => array(
'name' => 'Tag to search',
'exampleValue' => 'example',
'required' => true
),
));
public function collectData(){
$html = getContents(static::URI.'search.php?word=' . urlencode($this->getInput('tag')))
or returnClientError('Unable to query pixiv.net');
$regex = '/<input type="hidden"id="js-mount-point-search-result-list"data-items="([^"]*)/';
$timeRegex = '/img\/([0-9]{4})\/([0-9]{2})\/([0-9]{2})\/([0-9]{2})\/([0-9]{2})\/([0-9]{2})\//';
preg_match_all($regex, $html, $matches, PREG_SET_ORDER, 0);
if(!$matches) return;
$content = json_decode(html_entity_decode($matches[0][1]), true);
$count = 0;
foreach($content as $result) {
if($count == 10) break;
$count++;
$item = array();
$item['id'] = $result['illustId'];
$item['uri'] = 'https://www.pixiv.net/member_illust.php?mode=medium&illust_id=' . $result['illustId'];
$item['title'] = $result['illustTitle'];
$item['author'] = $result['userName'];
preg_match_all($timeRegex, $result['url'], $dt, PREG_SET_ORDER, 0);
$elementDate = DateTime::createFromFormat('YmdHis',
$dt[0][1] . $dt[0][2] . $dt[0][3] . $dt[0][4] . $dt[0][5] . $dt[0][6]);
$item['timestamp'] = $elementDate->getTimestamp();
$item['content'] = "<img src='" . $this->cacheImage($result['url'], $item['id']) . "' />";
$this->items[] = $item;
}
}
public function cacheImage($url, $illustId) {
$url = str_replace('_master1200', '', $url);
$url = str_replace('c/240x240/img-master/', 'img-original/', $url);
$path = CACHE_DIR . '/pixiv_img';
if(!is_dir($path))
mkdir($path, 0755, true);
if(!is_file($path . '/' . $illustId . '.jpeg')) {
$headers = array('Referer: https://www.pixiv.net/member_illust.php?mode=medium&illust_id=' . $illustId);
$illust = getContents($url, $headers);
if(strpos($illust, '404 Not Found') !== false) {
$illust = getContents(str_replace('jpg', 'png', $url), $headers);
}
file_put_contents($path . '/' . $illustId . '.jpeg', $illust);
}
return 'cache/pixiv_img/' . $illustId . '.jpeg';
}
}

View File

@@ -8,9 +8,9 @@ class RainbowSixSiegeBridge extends BridgeAbstract {
const DESCRIPTION = 'Latest articles from the Rainbow Six Siege blog';
public function collectData(){
$dlUrl = "https://prod-tridionservice.ubisoft.com/live/v1/News/Latest?templateId=tcm%3A152-7677";
$dlUrl .= "8-32&pageIndex=0&pageSize=10&language=en-US&detailPageId=tcm%3A152-194572-64";
$dlUrl .= "&keywordList=175426&siteId=undefined&useSeoFriendlyUrl=true";
$dlUrl = 'https://prod-tridionservice.ubisoft.com/live/v1/News/Latest?templateId=tcm%3A152-7677';
$dlUrl .= '8-32&pageIndex=0&pageSize=10&language=en-US&detailPageId=tcm%3A152-194572-64';
$dlUrl .= '&keywordList=175426&siteId=undefined&useSeoFriendlyUrl=true';
$jsonString = getContents($dlUrl) or returnServerError('Error while downloading the website content');
$json = json_decode($jsonString, true);

View File

@@ -25,7 +25,7 @@ class ReadComicsBridge extends BridgeAbstract {
return $timestamp;
}
$keywordsList = explode(";", $this->getInput('q'));
$keywordsList = explode(';', $this->getInput('q'));
foreach($keywordsList as $keywords) {
$html = $this->getSimpleHTMLDOM(self::URI . 'comic/' . rawurlencode($keywords))
or $this->returnServerError('Could not request readcomics.tv.');

View File

@@ -19,7 +19,7 @@ class ReporterreBridge extends BridgeAbstract {
// Replace all relative urls with absolute ones
$text = preg_replace(
'/(href|src)(\=[\"\'])(?!http)([^"\']+)/ims',
"$1$2" . self::URI . "$3",
'$1$2' . self::URI . '$3',
$text
);

View File

@@ -9,9 +9,9 @@ class Rue89Bridge extends FeedExpander {
protected function parseItem($item){
$item = parent::parseItem($item);
$url = "http://api.rue89.nouvelobs.com/export/mobile2/node/"
. str_replace(" ", "", substr($item['uri'], -8))
. "/full";
$url = 'http://api.rue89.nouvelobs.com/export/mobile2/node/'
. str_replace(' ', '', substr($item['uri'], -8))
. '/full';
$datas = json_decode(getContents($url), true);
$item['content'] = $datas['node']['body'];

View File

@@ -32,7 +32,7 @@ class SexactuBridge extends BridgeAbstract {
$item = array();
$item['author'] = self::AUTHOR;
$item['title'] = $title->plaintext;
$urlAttribute = "data-href";
$urlAttribute = 'data-href';
$uri = $title->$urlAttribute;
if($uri === false)
continue;

View File

@@ -73,7 +73,7 @@ class ShanaprojectBridge extends BridgeAbstract {
// Getting the picture is a little bit tricky as it is part of the style.
// Luckily the style is part of the parent div :)
if(preg_match("/url\(\/\/([^\)]+)\)/i", $anime->parent->style, $matches))
if(preg_match('/url\(\/\/([^\)]+)\)/i', $anime->parent->style, $matches))
return $matches[1];
returnServerError('Could not extract background image!');

View File

@@ -21,7 +21,7 @@ class Shimmie2Bridge extends DanbooruBridge {
protected function getItemFromElement($element){
$item = array();
$item['uri'] = $this->getURI() . $element->href;
$item['id'] = (int)preg_replace("/[^0-9]/", '', $element->getAttribute(static::IDATTRIBUTE));
$item['id'] = (int)preg_replace('/[^0-9]/', '', $element->getAttribute(static::IDATTRIBUTE));
$item['timestamp'] = time();
$thumbnailUri = $this->getURI() . $element->find('img', 0)->src;
$item['tags'] = $element->getAttribute('data-tags');

View File

@@ -14,7 +14,7 @@ class SoundCloudBridge extends BridgeAbstract {
)
));
const CLIENT_ID = '0aca19eae3843844e4053c6d8fdb7875';
const CLIENT_ID = '4jkoEFmZEDaqjwJ9Eih4ATNhcH3vMVfp';
public function collectData(){

57
bridges/SupInfoBridge.php Normal file
View File

@@ -0,0 +1,57 @@
<?php
class SupInfoBridge extends BridgeAbstract {
const MAINTAINER = 'teromene';
const NAME = 'SupInfoBridge';
const URI = 'https://www.supinfo.com';
const DESCRIPTION = 'Returns the newest articles.';
const PARAMETERS = array(array(
'tag' => array(
'name' => 'Category (not mandatory)',
'type' => 'text',
)
));
public function collectData() {
if(empty($this->getInput('tag'))) {
$html = getSimpleHTMLDOM(self::URI . '/articles/')
or returnServerError('Unable to fetch articles !');
} else {
$html = getSimpleHTMLDOM(self::URI . '/articles/tag/' . $this->getInput('tag'))
or returnServerError('Unable to fetch articles !');
}
$content = $html->find('#latest', 0)->find('ul[class=courseContent]', 0);
for($i = 0; $i < 5; $i++) {
$this->items[] = $this->fetchArticle($content->find('h4', $i)->find('a', 0)->href);
}
}
public function fetchArticle($link) {
$articleHTML = getSimpleHTMLDOM(self::URI . $link)
or returnServerError('Unable to fetch article !');
$article = $articleHTML->find('div[id=courseDocZero]', 0);
$item = array();
$item['author'] = $article->find('#courseMetas', 0)->find('a', 0)->plaintext;
$item['id'] = $link;
$item['uri'] = self::URI . $link;
$item['title'] = $article->find('h1', 0)->plaintext;
$date = explode(' ', $article->find('#courseMetas', 0)->find('span', 1)->plaintext);
$item['timestamp'] = DateTime::createFromFormat('d/m/Y H:i:s', $date[2] . ' ' . $date[4])->getTimestamp();
$article->find('div[id=courseHeader]', 0)->innertext = '';
$article->find('div[id=author-infos]', 0)->innertext = '';
$article->find('div[id=cartouche-tete]', 0)->innertext = '';
$item['content'] = $article;
return $item;
}
}

View File

@@ -0,0 +1,45 @@
<?php
class SuperSmashBlogBridge extends BridgeAbstract {
const MAINTAINER = 'corenting';
const NAME = 'Super Smash Blog';
const URI = 'https://www.smashbros.com/en_US/blog/index.html';
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = 'Latest articles from the Super Smash Blog blog';
public function collectData(){
$dlUrl = 'https://www.smashbros.com/data/bs/en_US/json/en_US.json';
$jsonString = getContents($dlUrl) or returnServerError('Error while downloading the website content');
$json = json_decode($jsonString, true);
foreach($json as $article) {
// Build content
$picture = $article['acf']['image1']['url'];
if (strlen($picture) != 0) {
$picture = str_get_html('<img src="https://www.smashbros.com/' . substr($picture, 8) . '"/>');
} else {
$picture = '';
}
$video = $article['acf']['link_url'];
if (strlen($video) != 0) {
$video = str_get_html('<a href="' . $video .'">Youtube video</a>');
} else {
$video = '';
}
$text = str_get_html($article['acf']['editor']);
$content = $picture . $video . $text;
// Build final item
$item = array();
$item['title'] = $article['title']['rendered'];
$item['timestamp'] = strtotime($article['date']);
$item['content'] = $content;
$item['uri'] = self::URI . '?post=' . $article['id'];
$this->items[] = $item;
}
}
}

View File

@@ -1,96 +0,0 @@
<?php
class T411Bridge extends BridgeAbstract {
const MAINTAINER = 'ORelio';
const NAME = 'T411 Bridge';
const URI = 'https://www.t411.al/';
const DESCRIPTION = 'Returns the 10 newest torrents with specified search
terms <br /> Use url part after "?" mark when using their search engine.';
const PARAMETERS = array( array(
'search' => array(
'name' => 'Search criteria',
'required' => true
)
));
public function collectData(){
//Utility function for retrieving text based on start and end delimiters
function extractFromDelimiters($string, $start, $end){
if(strpos($string, $start) !== false) {
$section_retrieved = substr($string, strpos($string, $start) + strlen($start));
$section_retrieved = substr($section_retrieved, 0, strpos($section_retrieved, $end));
return $section_retrieved;
}
return false;
}
//Retrieve torrent listing from search results, which does not contain torrent description
$url = self::URI
. 'torrents/search/?search='
. urlencode($this->getInput('search'))
. '&order=added&type=desc';
$html = getSimpleHTMLDOM($url)
or returnServerError('Could not request t411: ' . $url);
$results = $html->find('table.results', 0);
if (is_null($results))
returnServerError('No results from t411: ' . $url);
$limit = 0;
//Process each item individually
foreach($results->find('tr') as $element) {
//Limit total amount of requests and ignore table header
if($limit >= 10) {
break;
}
if(is_object($element->find('th', 0))) {
continue;
}
//Requests are rate-limited
usleep(500000); //So we need to wait (500ms)
//Retrieve data from RSS entry
$item_uri = self::URI
. 'torrents/details/?id='
. extractFromDelimiters($element->find('a.nfo', 0)->outertext, '?id=', '"');
$item_title = extractFromDelimiters($element->outertext, '" title="', '"');
$item_date = strtotime($element->find('dd', 0)->plaintext);
//Retrieve full description from torrent page
$item_html = getSimpleHTMLDOM($item_uri);
if(!$item_html) {
continue;
}
//Retrieve data from page contents
$item_desc = $item_html->find('div.description', 0);
$item_author = $item_html->find('a.profile', 0)->innertext;
//Cleanup advertisments
$divs = explode('<div class="align-center">', $item_desc->innertext);
$item_desc = '';
foreach ($divs as $text)
if (strpos($text, 'adprovider.adlure.net') === false)
$item_desc = $item_desc . '<div class="align-center">' . $text;
$item_desc = preg_replace('/<h2 class="align-center">LIENS DE T..?L..?CHARGEMENT<\/h2>/i', '', $item_desc);
//Build and add final item
$item = array();
$item['uri'] = $item_uri;
$item['title'] = $item_title;
$item['author'] = $item_author;
$item['timestamp'] = $item_date;
$item['content'] = $item_desc;
$this->items[] = $item;
$limit++;
}
}
}

View File

@@ -43,40 +43,261 @@ class VkBridge extends BridgeAbstract
or returnServerError('No results for group or user name "' . $this->getInput('u') . '".');
$text_html = iconv('windows-1251', 'utf-8', $text_html);
// makes album link generating work correctly
$text_html = str_replace('"class="page_album_link">', '" class="page_album_link">', $text_html);
$html = str_get_html($text_html);
$pageName = $html->find('.page_name', 0)->plaintext;
$this->pageName = $pageName;
$pageName = $html->find('.page_name', 0);
if (is_object($pageName)) {
$pageName = $pageName->plaintext;
$this->pageName = htmlspecialchars_decode($pageName);
}
$pinned_post_item = null;
$last_post_id = 0;
foreach ($html->find('.post') as $post) {
$is_pinned_post = false;
if (strpos($post->getAttribute('class'), 'post_fixed') !== false) {
$is_pinned_post = true;
}
if (is_object($post->find('a.wall_post_more', 0))) {
//delete link "show full" in content
$post->find('a.wall_post_more', 0)->outertext = '';
}
$content_suffix = '';
// looking for external links
$external_link_selectors = array(
'a.page_media_link_title',
'div.page_media_link_title > a',
'div.media_desc > a.lnk',
);
foreach($external_link_selectors as $sel) {
if (is_object($post->find($sel, 0))) {
$a = $post->find($sel, 0);
$innertext = $a->innertext;
$parsed_url = parse_url($a->getAttribute('href'));
if (strpos($parsed_url['path'], '/away.php') !== 0) continue;
parse_str($parsed_url['query'], $parsed_query);
$content_suffix .= "<br>External link: <a href='" . $parsed_query['to'] . "'>$innertext</a>";
}
}
// remove external link from content
$external_link_selectors_to_remove = array(
'div.page_media_thumbed_link',
'div.page_media_link_desc_wrap',
'div.media_desc > a.lnk',
);
foreach($external_link_selectors_to_remove as $sel) {
if (is_object($post->find($sel, 0))) {
$post->find($sel, 0)->outertext = '';
}
}
// looking for article
$article = $post->find('a.article_snippet', 0);
if (is_object($article)) {
if (strpos($article->getAttribute('class'), 'article_snippet_mini') !== false) {
$article_title_selector = 'div.article_snippet_mini_title';
$article_author_selector = 'div.article_snippet_mini_info > .mem_link,
div.article_snippet_mini_info > .group_link';
$article_thumb_selector = 'div.article_snippet_mini_thumb';
} else {
$article_title_selector = 'div.article_snippet__title';
$article_author_selector = 'div.article_snippet__author';
$article_thumb_selector = 'div.article_snippet__image';
}
$article_title = $article->find($article_title_selector, 0)->innertext;
$article_author = $article->find($article_author_selector, 0)->innertext;
$article_link = self::URI . ltrim($article->getAttribute('href'), '/');
$article_img_element_style = $article->find($article_thumb_selector, 0)->getAttribute('style');
preg_match('/background-image: url\((.*)\)/', $article_img_element_style, $matches);
if (count($matches) > 0) {
$content_suffix .= "<br><img src='" . $matches[1] . "'>";
}
$content_suffix .= "<br>Article: <a href='$article_link'>$article_title ($article_author)</a>";
$article->outertext = '';
}
// get video on post
$video = $post->find('div.post_video_desc', 0);
if (is_object($video)) {
$video_title = $video->find('div.post_video_title', 0)->plaintext;
$video_link = self::URI . ltrim( $video->find('a.lnk', 0)->getAttribute('href'), '/' );
$content_suffix .= "<br>Video: <a href='$video_link'>$video_title</a>";
$video->outertext = '';
}
// get all other videos
foreach($post->find('a.page_post_thumb_video') as $a) {
$video_title = $a->getAttribute('aria-label');
$temp = explode(' ', $video_title, 2);
if (count($temp) > 1) $video_title = $temp[1];
$video_link = self::URI . ltrim( $a->getAttribute('href'), '/' );
$content_suffix .= "<br>Video: <a href='$video_link'>$video_title</a>";
$a->outertext = '';
}
// get all photos
foreach($post->find('div.wall_text > a.page_post_thumb_wrap') as $a) {
$result = $this->getPhoto($a);
if ($result == null) continue;
$a->outertext = '';
$content_suffix .= "<br>$result";
}
// get albums
foreach($post->find('.page_album_wrap') as $el) {
$a = $el->find('.page_album_link', 0);
$album_title = $a->find('.page_album_title_text', 0)->getAttribute('title');
$album_link = self::URI . ltrim($a->getAttribute('href'), '/');
$el->outertext = '';
$content_suffix .= "<br>Album: <a href='$album_link'>$album_title</a>";
}
// get photo documents
foreach($post->find('a.page_doc_photo_href') as $a) {
$doc_link = self::URI . ltrim($a->getAttribute('href'), '/');
$doc_gif_label_element = $a->find('.page_gif_label', 0);
$doc_title_element = $a->find('.doc_label', 0);
if (is_object($doc_gif_label_element)) {
$gif_preview_img = backgroundToImg($a->find('.page_doc_photo', 0));
$content_suffix .= "<br>Gif: <a href='$doc_link'>$gif_preview_img</a>";
} else if (is_object($doc_title_element)) {
$doc_title = $doc_title_element->innertext;
$content_suffix .= "<br>Doc: <a href='$doc_link'>$doc_title</a>";
} else {
continue;
}
$a->outertext = '';
}
// get other documents
foreach($post->find('div.page_doc_row') as $div) {
$doc_title_element = $div->find('a.page_doc_title', 0);
if (is_object($doc_title_element)) {
$doc_title = $doc_title_element->innertext;
$doc_link = self::URI . ltrim($doc_title_element->getAttribute('href'), '/');
$content_suffix .= "<br>Doc: <a href='$doc_link'>$doc_title</a>";
} else {
continue;
}
$div->outertext = '';
}
// get polls
foreach($post->find('div.page_media_poll_wrap') as $div) {
$poll_title = $div->find('.page_media_poll_title', 0)->innertext;
$content_suffix .= "<br>Poll: $poll_title";
foreach($div->find('div.page_poll_text') as $poll_stat_title) {
$content_suffix .= '<br>- ' . $poll_stat_title->innertext;
}
$div->outertext = '';
}
// get sign
$post_author = $pageName;
foreach($post->find('a.wall_signed_by') as $a) {
$post_author = $a->innertext;
$a->outertext = '';
}
if (is_object($post->find('div.copy_quote', 0))) {
$copy_quote = $post->find('div.copy_quote', 0);
if ($copy_post_header = $copy_quote->find('div.copy_post_header', 0)) {
$copy_post_header->outertext = '';
}
$copy_quote_content = $copy_quote->innertext;
$copy_quote->outertext = "<br>Reposted: <br>$copy_quote_content";
}
$item = array();
$item['content'] = strip_tags(backgroundToImg($post->find('div.wall_text', 0)->innertext), '<br><img>');
if (is_object($post->find('a.page_media_link_title', 0))) {
$link = $post->find('a.page_media_link_title', 0)->getAttribute('href');
//external link in the post
$item['content'] .= "\n\rExternal link: "
. str_replace('/away.php?to=', '', urldecode($link));
}
//get video on post
if (is_object($post->find('span.post_video_title_content', 0))) {
$titleVideo = $post->find('span.post_video_title_content', 0)->plaintext;
$linkToVideo = self::URI . $post->find('a.page_post_thumb_video', 0)->getAttribute('href');
$item['content'] .= "\n\r {$titleVideo}: {$linkToVideo}";
}
$item['content'] .= $content_suffix;
// get post link
$item['uri'] = self::URI . $post->find('a.post_link', 0)->getAttribute('href');
$post_link = $post->find('a.post_link', 0)->getAttribute('href');
preg_match('/wall-?\d+_(\d+)/', $post_link, $preg_match_result);
$item['post_id'] = intval($preg_match_result[1]);
if (substr(self::URI, -1) == '/') {
$post_link = self::URI . ltrim($post_link, '/');
} else {
$post_link = self::URI . $post_link;
}
$item['uri'] = $post_link;
$item['timestamp'] = $this->getTime($post);
$item['author'] = $pageName;
$this->items[] = $item;
$item['title'] = $this->getTitle($item['content']);
$item['author'] = $post_author;
if ($is_pinned_post) {
// do not append it now
$pinned_post_item = $item;
} else {
$last_post_id = $item['post_id'];
$this->items[] = $item;
}
}
if (is_null($pinned_post_item)) {
return;
} else if (count($this->items) == 0) {
$this->items[] = $pinned_post_item;
} else if ($last_post_id < $pinned_post_item['post_id']) {
$this->items[] = $pinned_post_item;
usort($this->items, function ($item1, $item2) {
return $item2['post_id'] - $item1['post_id'];
});
}
}
private function getPhoto($a) {
$onclick = $a->getAttribute('onclick');
preg_match('/return showPhoto\(.+?({.*})/', $onclick, $preg_match_result);
if (count($preg_match_result) == 0) return;
$arg = htmlspecialchars_decode( str_replace('queue:1', '"queue":1', $preg_match_result[1]) );
$data = json_decode($arg, true);
if ($data == null) return;
$thumb = $data['temp']['base'] . $data['temp']['x_'][0] . '.jpg';
$original = '';
foreach(array('y_', 'z_', 'w_') as $key) {
if (!isset($data['temp'][$key])) continue;
if (!isset($data['temp'][$key][0])) continue;
if (substr($data['temp'][$key][0], 0, 4) == 'http') {
$base = '';
} else {
$base = $data['temp']['base'];
}
$original = $base . $data['temp'][$key][0] . '.jpg';
}
if ($original) {
return "<a href='$original'><img src='$thumb'></a>";
} else {
return "<img src='$thumb'>";
}
}
private function getTitle($content)
{
preg_match('/^["\w\ \p{Cyrillic}\(\)\?#«»-]+/mu', htmlspecialchars_decode($content), $result);
if (count($result) == 0) return 'untitled';
return $result[0];
}
private function getTime($post)

View File

@@ -18,10 +18,10 @@ class WhydBridge extends BridgeAbstract {
public function collectData(){
$html = '';
if(strlen(preg_replace("/[^0-9a-f]/", '', $this->getInput('u'))) == 24) {
if(strlen(preg_replace('/[^0-9a-f]/', '', $this->getInput('u'))) == 24) {
// is input the userid ?
$html = getSimpleHTMLDOM(
self::URI . 'u/' . preg_replace("/[^0-9a-f]/", '', $this->getInput('u'))
self::URI . 'u/' . preg_replace('/[^0-9a-f]/', '', $this->getInput('u'))
) or returnServerError('No results for this query.');
} else { // input may be the username
$html = getSimpleHTMLDOM(

View File

@@ -1,16 +1,12 @@
<?php
class WorldOfTanksBridge extends BridgeAbstract {
class WorldOfTanksBridge extends FeedExpander {
const MAINTAINER = 'mitsukarenai';
const MAINTAINER = 'Riduidel';
const NAME = 'World of Tanks';
const URI = 'http://worldoftanks.eu/';
const DESCRIPTION = 'News about the tank slaughter game.';
const PARAMETERS = array( array(
'category' => array(
// TODO: should be a list
'name' => 'nom de la catégorie'
),
'lang' => array(
'name' => 'Langue',
'type' => 'list',
@@ -26,47 +22,31 @@ class WorldOfTanksBridge extends BridgeAbstract {
)
));
private $title = '';
public function collectData() {
$this->collectExpandableDatas(sprintf('https://worldoftanks.eu/%s/rss/news/', $this->getInput('lang')));
}
public function getURI(){
if(!is_null($this->getInput('lang'))) {
$lang = $this->getInput('lang');
$uri = self::URI . $lang . '/news/';
if(!empty($this->getInput('category'))) {
$uri .= 'pc-browser/' . $this->getInput('category') . '/';
}
return $uri;
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
$item['content'] = $this->loadFullArticle($item['uri']);
return $item;
}
/**
* Loads the full article and returns the contents
* @param $uri The article URI
* @return The article content
*/
private function loadFullArticle($uri){
$html = getSimpleHTMLDOMCached($uri);
$content = $html->find('article', 0);
// Remove the scripts, please
foreach($content->find('script') as $script) {
$script->outertext = '';
}
return parent::getURI();
}
public function getName(){
return $this->title ?: parent::getName();
}
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request ' . $this->getURI());
debugMessage("loaded HTML from " . $this->getURI());
// customize name
$this->title = $html->find('title', 0)->innertext;
foreach($html->find('.b-imgblock_ico') as $infoLink) {
$this->parseLine($infoLink);
}
}
private function parseLine($infoLink){
$item = array();
$item['uri'] = self::URI . $infoLink->href;
// now load that uri from cache
debugMessage('loading page ' . $item['uri']);
$articlePage = getSimpleHTMLDOMCached($item['uri']);
$content = $articlePage->find('.l-content', 0);
defaultLinkTo($content, self::URI);
$item['title'] = $content->find('h1', 0)->innertext;
$item['content'] = $content->find('.b-content', 0)->innertext;
$item['timestamp'] = $content->find('.b-statistic_time', 0)->getAttribute("data-timestamp");
$this->items[] = $item;
return $content->innertext;
}
}

View File

@@ -0,0 +1,143 @@
<?php
/* This is a mashup of FlickrExploreBridge by sebsauvage and FlickrTagBridge
* by erwang.providing the functionality of both in one.
*/
class YGGTorrentBridge extends BridgeAbstract {
const MAINTAINER = 'teromene';
const NAME = 'Yggtorrent Bridge';
const URI = 'https://yggtorrent.is';
const DESCRIPTION = 'Returns torrent search from Yggtorrent';
const PARAMETERS = array(
array(
'cat' => array(
'name' => 'category',
'type' => 'list',
'values' => array(
'Toute les catégories' => 'all.all',
'Film/Vidéo - Toutes les sous-catégories' => '2145.all',
'Film/Vidéo - Animation' => '2145.2178',
'Film/Vidéo - Animation Série' => '2145.2179',
'Film/Vidéo - Concert' => '2145.2180',
'Film/Vidéo - Documentaire' => '2145.2181',
'Film/Vidéo - Émission TV' => '2145.2182',
'Film/Vidéo - Film' => '2145.2183',
'Film/Vidéo - Série TV' => '2145.2184',
'Film/Vidéo - Spectacle' => '2145.2185',
'Film/Vidéo - Sport' => '2145.2186',
'Film/Vidéo - Vidéo-clips' => '2145.2186',
'Audio - Toutes les sous-catégories' => '2139.all',
'Audio - Karaoké' => '2139.2147',
'Audio - Musique' => '2139.2148',
'Audio - Podcast Radio' => '2139.2150',
'Audio - Samples' => '2139.2149',
'Jeu vidéo - Toutes les sous-catégories' => '2142.all',
'Jeu vidéo - Autre' => '2142.2167',
'Jeu vidéo - Linux' => '2142.2159',
'Jeu vidéo - MacOS' => '2142.2160',
'Jeu vidéo - Microsoft' => '2142.2162',
'Jeu vidéo - Nintendo' => '2142.2163',
'Jeu vidéo - Smartphone' => '2142.2165',
'Jeu vidéo - Sony' => '2142.2164',
'Jeu vidéo - Tablette' => '2142.2166',
'Jeu vidéo - Windows' => '2142.2161',
'eBook - Toutes les sous-catégories' => '2140.all',
'eBook - Audio' => '2140.2151',
'eBook - Bds' => '2140.2152',
'eBook - Comics' => '2140.2153',
'eBook - Livres' => '2140.2154',
'eBook - Mangas' => '2140.2155',
'eBook - Presse' => '2140.2156',
'Emulation - Toutes les sous-catégories' => '2141.all',
'Emulation - Emulateurs' => '2141.2157',
'Emulation - Roms' => '2141.2158',
'GPS - Toutes les sous-catégories' => '2141.all',
'GPS - Applications' => '2141.2168',
'GPS - Cartes' => '2141.2169',
'GPS - Divers' => '2141.2170'
)
),
'nom' => array(
'name' => 'Nom',
'description' => 'Nom du torrent',
'type' => 'text'
),
'description' => array(
'name' => 'Description',
'description' => 'Description du torrent',
'type' => 'text'
),
'fichier' => array(
'name' => 'Fichier',
'description' => 'Fichier du torrent',
'type' => 'text'
),
'uploader' => array(
'name' => 'Uploader',
'description' => 'Uploader du torrent',
'type' => 'text'
),
)
);
public function collectData() {
$catInfo = explode('.', $this->getInput('cat'));
$category = $catInfo[0];
$subcategory = $catInfo[1];
$html = getSimpleHTMLDOM(self::URI . '/engine/search?name='
. $this->getInput('nom')
. '&description='
. $this->getInput('description')
. '&fichier='
. $this->getInput('fichier')
. '&file='
. $this->getInput('uploader')
. '&category='
. $category
. '&sub_category='
. $subcategory
. '&do=search')
or returnServerError('Unable to query Yggtorrent !');
$count = 0;
$results = $html->find('.results', 0);
if(!$results) return;
foreach($results->find('tr') as $row) {
$count++;
if($count == 1) continue;
if($count == 12) break;
$item = array();
$item['timestamp'] = $row->find('.hidden', 1)->plaintext;
$item['title'] = $row->find('a', 1)->plaintext;
$torrentData = $this->collectTorrentData($row->find('a', 1)->href);
$item['author'] = $torrentData['author'];
$item['content'] = $torrentData['content'];
$item['seeders'] = $row->find('td', 7)->plaintext;
$item['leechers'] = $row->find('td', 8)->plaintext;
$item['size'] = $row->find('td', 5)->plaintext;
$this->items[] = $item;
}
}
public function collectTorrentData($url) {
//For weird reason, the link we get can be invalid, we fix it.
$url_full = explode('/', $url);
$url_full[4] = urlencode($url_full[4]);
$url_full[5] = urlencode($url_full[5]);
$url_full[6] = urlencode($url_full[6]);
$url = implode('/', $url_full);
$page = getSimpleHTMLDOM($url) or returnServerError('Unable to query Yggtorrent page !');
$author = $page->find('.informations', 0)->find('a', 4)->plaintext;
$content = $page->find('.default', 1);
return array('author' => $author, 'content' => $content);
}
}

View File

@@ -25,14 +25,14 @@ class YoutubeBridge extends BridgeAbstract {
'By channel id' => array(
'c' => array(
'name' => 'channel id',
'exampleValue' => "15",
'exampleValue' => '15',
'required' => true
)
),
'By playlist Id' => array(
'p' => array(
'name' => 'playlist id',
'exampleValue' => "15"
'exampleValue' => '15'
)
),
'Search result' => array(
@@ -195,7 +195,7 @@ class YoutubeBridge extends BridgeAbstract {
$this->request = $this->getInput('s');
$page = 1;
if($this->getInput('pa'))
$page = (int)preg_replace("/[^0-9]/", '', $this->getInput('pa'));
$page = (int)preg_replace('/[^0-9]/', '', $this->getInput('pa'));
$url_listing = self::URI
. 'results?search_query='
@@ -207,7 +207,7 @@ class YoutubeBridge extends BridgeAbstract {
$html = $this->ytGetSimpleHTMLDOM($url_listing)
or returnServerError("Could not request YouTube. Tried:\n - $url_listing");
$this->ytBridgeParseHtmlListing($html, 'div.yt-lockup', 'h3');
$this->ytBridgeParseHtmlListing($html, 'div.yt-lockup', 'h3 > a');
$this->feedName = 'Search: ' . str_replace(' - YouTube', '', $html->find('title', 0)->plaintext); // feedName will be used by getName()
} else { /* no valid mode */
returnClientError("You must either specify either:\n - YouTube
@@ -226,5 +226,5 @@ class YoutubeBridge extends BridgeAbstract {
default:
return parent::getName();
}
}
}
}

55
bridges/ZenodoBridge.php Normal file
View File

@@ -0,0 +1,55 @@
<?php
class ZenodoBridge extends BridgeAbstract {
const MAINTAINER = 'theradialactive';
const NAME = 'Zenodo';
const URI = 'https://zenodo.org';
const CACHE_TIMEOUT = 10;
const DESCRIPTION = 'Returns the newest content of Zenodo';
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('zenodo.org not reachable.');
foreach($html->find('div.record-elem') as $element) {
$item = array();
$item['uri'] = self::URI . $element->find('h4', 0)->find('a', 0)->href;
$item['title'] = trim(
htmlspecialchars_decode($element->find('h4', 0)->find('a', 0)->innertext,
ENT_QUOTES
)
);
foreach($element->find('p', 0)->find('span') as $authors) {
$item['author'] = $item['author'] . $authors . '; ';
}
$content = $element->find('p.hidden-xs', 0)->find('a', 0)->innertext . '<br>';
$type = '<br>Type: ' . $element->find('span.label-default', 0)->innertext;
$raw_date = $element->find('small.text-muted', 0)->innertext;
$clean_date = date_parse(str_replace('Uploaded on ', '', $raw_date));
$content = $content . date_parse($clean_date);
$item['timestamp'] = mktime(
$clean_date['hour'],
$clean_date['minute'],
$clean_date['second'],
$clean_date['month'],
$clean_date['day'],
$clean_date['year']
);
$access = '';
if ($element->find('span.label-success', 0)->innertext) {
$access = 'Open Access';
} elseif ($element->find('span.label-warning', 0)->innertext) {
$access = 'Embargoed Access';
} else {
$access = $element->find('span.label-error', 0)->innertext;
}
$access = '<br>Access: ' . $access;
$publication = '<br>Publication Date: ' . $element->find('span.label-info', 0)->innertext;
$item['content'] = $content . $type . $access . $publication;
$this->items[] = $item;
}
}
}

View File

@@ -8,7 +8,9 @@ class FileCache implements CacheInterface {
protected $param;
public function loadData(){
return unserialize(file_get_contents($this->getCacheFile()));
if(file_exists($this->getCacheFile())) {
return unserialize(file_get_contents($this->getCacheFile()));
}
}
public function saveData($datas){
@@ -17,7 +19,7 @@ class FileCache implements CacheInterface {
$writeStream = file_put_contents($this->getCacheFile(), serialize($datas));
if($writeStream === false) {
throw new \Exception("Cannot write the cache... Do you have the right permissions ?");
throw new \Exception('Cannot write the cache... Do you have the right permissions ?');
}
return $this;

44
config.default.ini.php Normal file
View File

@@ -0,0 +1,44 @@
; <?php exit; ?> DO NOT REMOVE THIS LINE
; This file contains the default settings for RSS-Bridge. Do not change this
; file, it will be replaced on the next update of RSS-Bridge! You can specify
; your own configuration in 'config.ini.php' (copy this file).
[cache]
; Allow users to specify custom timeout for specific requests.
; true = enabled
; false = disabled (default)
custom_timeout = false
[proxy]
; Sets the proxy url (i.e. "tcp://192.168.0.0:32")
; "" = Proxy disabled (default)
url = ""
; Sets the proxy name that is shown on the bridge instead of the proxy url.
; "" = Show proxy url
name = "Hidden proxy name"
; Allow users to disable proxy usage for specific requests.
; true = enabled
; false = disabled (default)
by_bridge = false
[authentication]
; Enables authentication for all requests to this RSS-Bridge instance.
;
; Warning: You'll have to upgrade existing feeds after enabling this option!
;
; true = enabled
; false = disabled (default)
enable = false
; The username for authentication. Insert this name when prompted for login.
username = ""
; The password for authentication. Insert this password when prompted for login.
; Use a strong password to prevent others from guessing your login!
password = ""

View File

@@ -15,8 +15,11 @@ class AtomFormat extends FormatAbstract{
$extraInfos = $this->getExtraInfos();
$title = $this->xml_encode($extraInfos['name']);
$uri = !empty($extraInfos['uri']) ? $extraInfos['uri'] : 'https://github.com/sebsauvage/rss-bridge';
$icon = $this->xml_encode('http://icons.better-idea.org/icon?url='. $uri .'&size=64');
$uri = !empty($extraInfos['uri']) ? $extraInfos['uri'] : 'https://github.com/RSS-Bridge/rss-bridge';
$uriparts = parse_url($uri);
$icon = $this->xml_encode($uriparts['scheme'] . '://' . $uriparts['host'] .'/favicon.ico');
$uri = $this->xml_encode($uri);
$entries = '';
@@ -37,6 +40,16 @@ class AtomFormat extends FormatAbstract{
}
}
$entryCategories = '';
if(isset($item['categories'])) {
foreach($item['categories'] as $category) {
$entryCategories .= '<category term="'
. $this->xml_encode($category)
. '"/>'
. PHP_EOL;
}
}
$entries .= <<<EOD
<entry>
@@ -49,6 +62,7 @@ class AtomFormat extends FormatAbstract{
<updated>{$entryTimestamp}</updated>
<content type="html">{$entryContent}</content>
{$entryEnclosures}
{$entryCategories}
</entry>
EOD;

View File

@@ -47,6 +47,20 @@ class HtmlFormat extends FormatAbstract {
$entryEnclosures .= '</div>';
}
$entryCategories = '';
if(isset($item['categories'])) {
$entryCategories = '<div class="categories"><p>Categories:</p>';
foreach($item['categories'] as $category) {
$entryCategories .= '<li class="category">'
. $this->sanitizeHtml($category)
. '</li>';
}
$entryCategories .= '</div>';
}
$entries .= <<<EOD
<section class="feeditem">
@@ -55,6 +69,7 @@ class HtmlFormat extends FormatAbstract {
{$entryAuthor}
{$entryContent}
{$entryEnclosures}
{$entryCategories}
</section>
EOD;

View File

@@ -18,10 +18,11 @@ class MrssFormat extends FormatAbstract {
if(!empty($extraInfos['uri'])) {
$uri = $this->xml_encode($extraInfos['uri']);
} else {
$uri = 'https://github.com/sebsauvage/rss-bridge';
$uri = 'https://github.com/RSS-Bridge/rss-bridge';
}
$icon = $this->xml_encode('http://icons.better-idea.org/icon?url='. $uri .'&size=64');
$uriparts = parse_url($uri);
$icon = $this->xml_encode($uriparts['scheme'] . '://' . $uriparts['host'] .'/favicon.ico');
$items = '';
foreach($this->getItems() as $item) {
@@ -50,6 +51,16 @@ Some media files might not be shown to you. Consider using the ATOM format inste
}
}
$entryCategories = '';
if(isset($item['categories'])) {
foreach($item['categories'] as $category) {
$entryCategories .= '<category>'
. $category . '</category>'
. PHP_EOL;
}
}
$items .= <<<EOD
<item>
@@ -60,6 +71,7 @@ Some media files might not be shown to you. Consider using the ATOM format inste
<description>{$itemContent}{$entryEnclosuresWarning}</description>
<author>{$itemAuthor}</author>
{$entryEnclosures}
{$entryCategories}
</item>
EOD;

View File

@@ -1,37 +1,21 @@
<?php
/*
TODO :
- factorize the annotation system
- factorize to adapter : Format, Bridge, Cache(actually code is almost the same)
- implement annotation cache for entrance page
- Cache : I think logic must be change as least to avoid to reconvert object from json in FileCache case.
- add namespace to avoid futur problem ?
- see FIXME mentions in the code
- implement header('X-Cached-Version: '.date(DATE_ATOM, filemtime($cachefile)));
*/
require_once __DIR__ . '/lib/RssBridge.php';
// Defines the minimum required PHP version for RSS-Bridge
define('PHP_VERSION_REQUIRED', '5.6.0');
//define('PROXY_URL', 'tcp://192.168.0.0:28');
// Set to true if you allow users to disable proxy usage for specific bridges
define('PROXY_BYBRIDGE', false);
// Comment this line or keep PROXY_NAME empty to display PROXY_URL instead
define('PROXY_NAME', 'Hidden Proxy Name');
// Allows the operator to specify custom cache timeouts via '&_cache_timeout=3600'
// true: enabled, false: disabled (default)
define('CUSTOM_CACHE_TIMEOUT', false);
date_default_timezone_set('UTC');
error_reporting(0);
// Specify directory for cached files (using FileCache)
define('CACHE_DIR', __DIR__ . '/cache');
// Specify path for whitelist file
define('WHITELIST_FILE', __DIR__ . '/whitelist.txt');
Configuration::verifyInstallation();
Configuration::loadConfiguration();
Authentication::showPromptIfNeeded();
date_default_timezone_set('UTC');
error_reporting(0);
/*
Move the CLI arguments to the $_GET array, in order to be able to use
@@ -61,40 +45,6 @@ if(file_exists('DEBUG')) {
}
}
require_once __DIR__ . '/lib/RssBridge.php';
// Check PHP version
if(version_compare(PHP_VERSION, PHP_VERSION_REQUIRED) === -1)
die('RSS-Bridge requires at least PHP version ' . PHP_VERSION_REQUIRED . '!');
// extensions check
if(!extension_loaded('openssl'))
die('"openssl" extension not loaded. Please check "php.ini"');
if(!extension_loaded('libxml'))
die('"libxml" extension not loaded. Please check "php.ini"');
if(!extension_loaded('mbstring'))
die('"mbstring" extension not loaded. Please check "php.ini"');
if(!extension_loaded('simplexml'))
die('"simplexml" extension not loaded. Please check "php.ini"');
if(!extension_loaded('curl'))
die('"curl" extension not loaded. Please check "php.ini"');
// configuration checks
if(ini_get('allow_url_fopen') !== "1")
die('"allow_url_fopen" is not set to "1". Please check "php.ini');
// Check cache folder permissions (write permissions required)
if(!is_writable(CACHE_DIR))
die('RSS-Bridge does not have write permissions for ' . CACHE_DIR . '!');
// Check whitelist file permissions (only in DEBUG mode)
if(!file_exists(WHITELIST_FILE) && !is_writable(dirname(WHITELIST_FILE)))
die('RSS-Bridge does not have write permissions for ' . WHITELIST_FILE . '!');
// FIXME : beta test UA spoofing, please report any blacklisting by PHP-fopen-unfriendly websites
$userAgent = 'Mozilla/5.0(X11; Linux x86_64; rv:30.0)';
@@ -303,7 +253,8 @@ EOD;
echo $inactiveBridges;
?>
<section class="footer">
<a href="https://github.com/RSS-Bridge/rss-bridge">RSS-Bridge 2018-04-06 ~ Public Domain</a><br />
<a href="https://github.com/RSS-Bridge/rss-bridge">RSS-Bridge ~ Public Domain</a><br />
<p class="version"> <?= Configuration::getVersion() ?> </p>
<?= $activeFoundBridgeCount; ?>/<?= count($bridgeList) ?> active bridges. <br />
<?php
if($activeFoundBridgeCount !== count($bridgeList)) {

31
lib/Authentication.php Normal file
View File

@@ -0,0 +1,31 @@
<?php
class Authentication {
public static function showPromptIfNeeded() {
if(Configuration::getConfig('authentication', 'enable') === true) {
if(!Authentication::verifyPrompt()) {
header('WWW-Authenticate: Basic realm="RSS-Bridge"');
header('HTTP/1.0 401 Unauthorized');
die('Please authenticate in order to access this instance !');
}
}
}
public static function verifyPrompt() {
if(isset($_SERVER['PHP_AUTH_USER']) && isset($_SERVER['PHP_AUTH_PW'])) {
if(Configuration::getConfig('authentication', 'username') === $_SERVER['PHP_AUTH_USER']
&& Configuration::getConfig('authentication', 'password') === $_SERVER['PHP_AUTH_PW']) {
return true;
} else {
error_log('[RSS-Bridge] Failed authentication attempt from ' . $_SERVER['REMOTE_ADDR']);
}
}
return false;
}
}

120
lib/Configuration.php Normal file
View File

@@ -0,0 +1,120 @@
<?php
class Configuration {
public static $VERSION = '2018-07-17';
public static $config = null;
public static function verifyInstallation() {
// Check PHP version
if(version_compare(PHP_VERSION, PHP_VERSION_REQUIRED) === -1)
die('RSS-Bridge requires at least PHP version ' . PHP_VERSION_REQUIRED . '!');
// extensions check
if(!extension_loaded('openssl'))
die('"openssl" extension not loaded. Please check "php.ini"');
if(!extension_loaded('libxml'))
die('"libxml" extension not loaded. Please check "php.ini"');
if(!extension_loaded('mbstring'))
die('"mbstring" extension not loaded. Please check "php.ini"');
if(!extension_loaded('simplexml'))
die('"simplexml" extension not loaded. Please check "php.ini"');
if(!extension_loaded('curl'))
die('"curl" extension not loaded. Please check "php.ini"');
// Check cache folder permissions (write permissions required)
if(!is_writable(CACHE_DIR))
die('RSS-Bridge does not have write permissions for ' . CACHE_DIR . '!');
// Check whitelist file permissions (only in DEBUG mode)
if(!file_exists(WHITELIST_FILE) && !is_writable(dirname(WHITELIST_FILE)))
die('RSS-Bridge does not have write permissions for ' . WHITELIST_FILE . '!');
}
public static function loadConfiguration() {
if(!file_exists('config.default.ini.php'))
die('The default configuration file "config.default.ini.php" is missing!');
Configuration::$config = parse_ini_file('config.default.ini.php', true, INI_SCANNER_TYPED);
if(!Configuration::$config)
die('Error parsing config.default.ini.php');
if(file_exists('config.ini.php')) {
// Replace default configuration with custom settings
foreach(parse_ini_file('config.ini.php', true, INI_SCANNER_TYPED) as $header => $section) {
foreach($section as $key => $value) {
// Skip unknown sections and keys
if(array_key_exists($header, Configuration::$config) && array_key_exists($key, Configuration::$config[$header])) {
Configuration::$config[$header][$key] = $value;
}
}
}
}
if(!is_string(self::getConfig('proxy', 'url')))
die('Parameter [proxy] => "url" is not a valid string! Please check "config.ini.php"!');
if(!empty(self::getConfig('proxy', 'url')))
define('PROXY_URL', self::getConfig('proxy', 'url'));
if(!is_bool(self::getConfig('proxy', 'by_bridge')))
die('Parameter [proxy] => "by_bridge" is not a valid Boolean! Please check "config.ini.php"!');
define('PROXY_BYBRIDGE', self::getConfig('proxy', 'by_bridge'));
if(!is_string(self::getConfig('proxy', 'name')))
die('Parameter [proxy] => "name" is not a valid string! Please check "config.ini.php"!');
define('PROXY_NAME', self::getConfig('proxy', 'name'));
if(!is_bool(self::getConfig('cache', 'custom_timeout')))
die('Parameter [cache] => "custom_timeout" is not a valid Boolean! Please check "config.ini.php"!');
define('CUSTOM_CACHE_TIMEOUT', self::getConfig('cache', 'custom_timeout'));
if(!is_bool(self::getConfig('authentication', 'enable')))
die('Parameter [authentication] => "enable" is not a valid Boolean! Please check "config.ini.php"!');
if(!is_string(self::getConfig('authentication', 'username')))
die('Parameter [authentication] => "username" is not a valid string! Please check "config.ini.php"!');
if(!is_string(self::getConfig('authentication', 'password')))
die('Parameter [authentication] => "password" is not a valid string! Please check "config.ini.php"!');
}
public static function getConfig($category, $key) {
if(array_key_exists($category, self::$config) && array_key_exists($key, self::$config[$category])) {
return self::$config[$category][$key];
}
return null;
}
public static function getVersion() {
$headFile = '.git/HEAD';
if(file_exists($headFile)) {
$revisionHashFile = '.git/' . substr(file_get_contents($headFile), 5, -1);
$branchName = explode('/', $revisionHashFile)[3];
if(file_exists($revisionHashFile)) {
return 'git.' . $branchName . '.' . substr(file_get_contents($revisionHashFile), 0, 7);
}
}
return Configuration::$VERSION;
}
}

View File

@@ -64,7 +64,10 @@ function buildBridgeException($e, $bridge){
$body = 'Error message: `'
. $e->getMessage()
. "`\nQuery string: `"
. $_SERVER['QUERY_STRING'] . '`';
. $_SERVER['QUERY_STRING']
. "`\nVersion: `"
. Configuration::getVersion()
. '`';
$link = buildGitHubIssueQuery($title, $body, 'bug report', $bridge->getMaintainer());

View File

@@ -24,15 +24,15 @@ abstract class FeedExpander extends BridgeAbstract {
switch(true) {
case isset($rssContent->item[0]):
debugMessage('Detected RSS 1.0 format');
$this->feedType = "RSS_1_0";
$this->feedType = 'RSS_1_0';
break;
case isset($rssContent->channel[0]):
debugMessage('Detected RSS 0.9x or 2.0 format');
$this->feedType = "RSS_2_0";
$this->feedType = 'RSS_2_0';
break;
case isset($rssContent->entry[0]):
debugMessage('Detected ATOM format');
$this->feedType = "ATOM_1_0";
$this->feedType = 'ATOM_1_0';
break;
default:
debugMessage('Unknown feed format/version');

View File

@@ -14,6 +14,8 @@ require __DIR__ . '/Bridge.php';
require __DIR__ . '/BridgeAbstract.php';
require __DIR__ . '/FeedExpander.php';
require __DIR__ . '/Cache.php';
require __DIR__ . '/Authentication.php';
require __DIR__ . '/Configuration.php';
require __DIR__ . '/validation.php';
require __DIR__ . '/html.php';

View File

@@ -22,10 +22,12 @@ function getContents($url, $header = array(), $opts = array()){
}
$content = curl_exec($ch);
$curlError = curl_error($ch);
$curlErrno = curl_errno($ch);
curl_close($ch);
if($content === false)
debugMessage('Cant\'t download ' . $url);
debugMessage('Cant\'t download ' . $url . ' cUrl error: ' . $curlError . ' (' . $curlErrno . ')');
return $content;
}

View File

@@ -26,7 +26,7 @@ EOD;
$bridge = Bridge::create($bridgeName);
if($bridge == false)
return "";
return '';
$HTTPSWarning = '';
if(strpos($bridge->getURI(), 'https') !== 0) {

View File

@@ -70,4 +70,8 @@
<rule ref="Squiz.WhiteSpace.SemicolonSpacing"/>
<!-- Do not add whitespace at start or end of a file or end of a line -->
<rule ref="Squiz.WhiteSpace.SuperfluousWhitespace"/>
<!-- Whenever possible use single quote strings -->
<rule ref="Squiz.Strings.DoubleQuoteUsage">
<exclude name="Squiz.Strings.DoubleQuoteUsage.ContainsVar" />
</rule>
</ruleset>

View File

@@ -5,7 +5,6 @@ html, body, div, span, applet, object, iframe, h1, h2, h3, h4, h5, h6, p, blockq
border: 0;
outline: 0;
font-size: 100%;
font: inherit;
vertical-align: baseline;
}
@@ -70,12 +69,14 @@ a.backlink, a.backlink:link, a.backlink:visited, a.itemtitle, a.itemtitle:link,
}
section > div.content, section > div.categories,
section > div.content, section > div.attachments {
padding: 10px;
}
section > div.categories > li.category,
section > div.attachments > li.enclosure {
list-style-type: circle;
@@ -111,3 +112,8 @@ button.backbutton, button.rss-feed {
}
img {
max-width: 100%;
}

View File

@@ -143,6 +143,11 @@ section.footer:hover {
}
section.footer .version {
font-size: 80%;
}
section > h2 {