1
0
mirror of https://github.com/RSS-Bridge/rss-bridge.git synced 2025-08-16 21:44:01 +02:00

Compare commits

..

360 Commits

Author SHA1 Message Date
logmanoriginal
6a98293fb3 [Configuration] Bump version to 2018-07-17 2018-07-17 20:44:01 +02:00
logmanoriginal
d79630e3b8 [Configuration] Remove check for allow_url_fopen
This commit follows the changes done in commits

fbf874cb29
ead7b2e8de
2018-07-17 20:39:14 +02:00
teromene
1f2fe25471 Fix LeBonCoinBridge, now uses getContents correctly, 2018-07-17 10:50:30 +02:00
Antoine Cadoret
87fc9e9156 fix LeBonCoin bridge (#747) 2018-07-16 20:13:08 +02:00
Nemo
c7b0c9fd31 Amazon Price Tracker Bridge (#741)
* [amazonprice] Adds AmazonPriceTracker bridge
2018-07-16 14:54:52 +02:00
Teromene
fbf874cb29 Update README.md
Remove allow_url_fopen requirement. This should no longer be necessary. Added requirement for curl.
2018-07-16 12:37:09 +02:00
Eugene Molotov
049ee52fb5 Implemented feed item categories (#746) 2018-07-16 12:32:24 +02:00
TheRadialActive
3f41d0593a Added RSS bridge for zenodo.org (#749)
* added RSS bridge for zenodo.org
2018-07-16 12:02:41 +02:00
sysadminstory
7126f5e838 [DealabsBridge] First version of the generic "Pepper" Bridge (#726)
* [DealabsBridge] First version of the generic "Pepper" Bridge
2018-07-13 00:35:13 +01:00
Nemo
ead7b2e8de [fb2] Switches to getContents (#742) 2018-07-10 02:29:47 +01:00
LogMANOriginal
0d80a19e84 [FacebookBridge] Add context for public Facebook groups (#739)
The previous context is now labeled 'User', while the new context is
labeled 'Group'. The existing code was not changed, instead new group*
functions were implemented to handle groups.

The general principle of capturing groups is the same as done for users
with adjustments to account for different HTML structures.

Captcha responses are currently not supported for groups! There doesn't
seem to be a way to trigger them consistently, which makes it hard to
handle them properly.

Features of the group context:

- The feed title is based on the group name
- The group URI used for capturing is returned for the feed URI
- Author names and timestamps are reproduced from the source
- Post titles are reproduced from the source if they exist, otherwise
the title is build manually from the author name and the content
- Original contents are included with the feed
- All images are attached as enclosures as well

Closes #
2018-07-08 17:16:00 +02:00
logmanoriginal
42c699f474 formats: Fix favicon not found if url contains path 2018-06-30 10:27:05 +02:00
logmanoriginal
2bc8daa101 [JustETFBridge] Add new bridge
Supports latest news and profiling a given ETF in Englisch, German
or Italian language. Cover images are attached as enclosures and not
as part of the content.

News:

Optionally loads the full article for each news item. Some articles
may include scripts to provide interactive graphs. These scripts are
removed as they would be rendered as pure text and a message is shown
instead: "[Content removed! Visit site to see full contents!]"

Profile:

Optionally includes the ETF strategy and description.
2018-06-30 10:27:05 +02:00
logmanoriginal
bca79d3f88 [KununuBridge] Fix broken page layout and sort reviews 2018-06-30 10:27:05 +02:00
logmanoriginal
90dc968fd1 Fix PHPCS error 2018-06-30 10:26:48 +02:00
Teromene
da6b98851c Add recuperation of the current version from git if available (#731)
* Add recuperation of the current version from git if available
* Include version when auto-reporting an error
2018-06-30 10:24:22 +02:00
teromene
71c29d4192 Fix phpcs for master. 2018-06-29 23:15:22 +01:00
LogMANOriginal
193ca87afa [phpcs] enforce single quotes (#732)
* [phpcs] Add rule to enforce single quoted strings
2018-06-29 22:55:33 +01:00
Nemo
5ea79ac1fc Add markdown support to Container Linux Feed (#730) 2018-06-28 20:54:42 +02:00
Teromene
937ea49271 Add basic authentication support (#728)
* Move configuration in its own class in order to reduce the verbosity of index.php
* Add authentication mechanism using HTTP auth
* Add a method to get the config parameters
* Remove the installation checks from the index page
* Log all failed authentication attempts
2018-06-27 19:09:41 +02:00
logmanoriginal
95686b803c [IsoHuntBridge] Remove bridge
isoHunt has discontinued services due to legal reasons and is now
accessible via https://isohunts.to

While it is certainly possible to rewrite the bridge to fetch some
information from the new site, it wouldn't be able to provide as
much functionality as before. This is due to isoHunt having removed
all searching and filtering options, only providing static HTML pages
for general categories (anime, movies, etc...). Those pages, however,
are heavily broken.

Unless someone is interested in monitoring the general categories
the effort of upgrading the bridge to the new site is not worth taking
time for.

Users of isoHunt are asked to make use of their client application,
as they don't provide online services anymore (it's now in the darknet)

Here is the statement from isoHunt:

"Due to hard regulations and security issues for bittorrent users, we
have moved into a more secure and even faster district of the internet!

[...]

Torrent Downloads have a high risk of getting legal problems. That is
why we do not offer torrentfiles any more. [...]"

-- source: https://isohunts.to
2018-06-24 18:33:50 +02:00
logmanoriginal
5087f5f79e [FacebookBridge] Support facebook links as user name
Allows users to paste facebook links as user name. The link must contain
the correct host (www.facebook.com) and a valid path (/user-name/...).
The first part of the path is used for the user name. Errors are returned
in case something went wrong.

References #706
2018-06-24 11:14:08 +02:00
logmanoriginal
4a5f190e0e [FacebookBridge] Add option to skip reviews
Reviews are provided the same way as summary posts and therefore returned
as separate feed item for each review. This commit adds a new option
'&skip_reviews=on' to skip reviews entirely.

References #706
2018-06-24 10:52:22 +02:00
logmanoriginal
01a2746715 [YoutubeBridge] Fix sniff violation
This is a fix for a sniff violation not detected by newer versions
of phpcs (not sure why though, it's detected in version 2.7.1).
2018-06-23 21:28:30 +02:00
Nemo
f4a60c1777 Add dockerfile to create an official docker image (#720) 2018-06-23 16:51:48 +02:00
sysadminstory
1b08bce779 [DealabsBridge] Follow site changes (#721)
- Changed some CSS class to follow the website changes (again)
2018-06-21 13:14:59 +01:00
Nemo
9fa74a36c6 Adds Container Linux releases RSS Feed (#718)
* Adds Container Linux releases RSS Feed
2018-06-19 19:39:08 +01:00
Corentin Garcia
7493e2b5b8 [GrandComicsDatabaseBridge] Add bridge (#717)
closes #709
2018-06-15 21:09:09 +02:00
Corentin Garcia
8e468a9ca7 [SuperSmashBlogBridge] Added bridge (#716) 2018-06-15 21:05:31 +02:00
Joe Digilio
50924b9213 Abort on parse error of config.default.ini.php (#714)
If there is an error parsing the default config file, then abort.
2018-06-15 21:02:06 +02:00
logmanoriginal
4c5013bc82 [index] Bump release version to 2018-06-10 2018-06-10 22:14:58 +02:00
Eugene Molotov
7dc09db9ca [VkBridge] More beatifications and fixes (#712)
* Add one more selector for article_author_selector
* Extend video parsing
* Add poll parsing
2018-06-10 22:09:50 +02:00
hunhejj
d92da8f0f7 Add cUrl error message and code to the debugMessage (#711) 2018-06-10 22:08:45 +02:00
logmanoriginal
064ba456e8 [InstagramBridge] Fix broken compatibility for media_type parameter
The media_type parameter was recently replaced by media_type_u (for
user mode) and media_type_h (for hashtag mode). This was necessary
in order to add the media type 'story' only for the user mode.

"The reason for that is that RSS-Bridge supports multiple parameters
with the same name if and only if they contain the exact same value.
Here, hashtags don't have stories, so it would not be possible to
pass "story" as a parameter. This is a design mistake that I made
when I added support for hashtags."

-- 8770c87389 (r28871502)

However as pointed out this change breaks existing feeds as the
parameter name is no longer compatible to previous implementations.

This commit changes the implementation to provide the old media_type
parameter globally and check for invalid options on each request. If
a user uses the 'story' option in history mode the bridge returns a
client error.

references 8770c87
references #694
fixes #696
fixes #699
fixes #701
2018-05-29 12:52:31 +02:00
LogMANOriginal
8ac8e08abf Add user config (#653)
Uses the parse_ini_file function to load default settings from the default configuration file 'config.default.ini.php'. Optionally loads custom settings from 'config.ini.php' to replace the default
values.
2018-05-29 11:52:17 +02:00
rogerdc
c4f32c31a8 Add ChristianDailyReporterBridge (#697) 2018-05-29 11:28:22 +02:00
Eugene Molotov
4369e077c2 [VkBridge] Fixed image src link generating for photo (#700) 2018-05-29 11:01:54 +02:00
sysadminstory
1045850043 [DealabsBridge] Follow site changes, fix unhandled case (#703)
* [DealabsBridge] Follow site changes, fix unhandled case

- Fixed the case where no discount was shown
- Changed some CSS class to follow the website changes
2018-05-29 10:52:13 +02:00
teromene
2d8f4dc3c5 Fix space in URL resulting in API errors. 2018-05-05 18:10:19 +01:00
teromene
779b638fb4 Added ElloBridge. Closes #683 2018-05-05 18:06:27 +01:00
teromene
3ca59392c2 Fix for crashes when accessing FileCache in case it has been purged/not created yet. 2018-05-05 18:05:48 +01:00
teromene
9b34b68180 Do not use an external service in order to fetch the favicon. 2018-05-05 13:55:38 +01:00
teromene
79ebdc4b39 Warn the user when trying to fetch a non-public facebook page. 2018-05-05 13:49:49 +01:00
teromene
8770c87389 Added support for stories in InstagramBridge. Closes #665
Renamed parameters as stories are only available in user mode.
Use a regex instead of HTML parsing to extract the JSON, as it is way faster.
2018-05-05 13:00:59 +01:00
Eugene Molotov
c1e3352218 [VkBridge] Extended article link parsing (#685)
* [VkBridge] Extended article link parsing
2018-05-05 12:03:54 +02:00
Grégory T
00570ce1b4 [ETTVBridge] New bridge, first push (#680)
* [ETTVBridge] New bridge
2018-04-30 23:18:39 +02:00
teromene
df33dcff4e [YGGTorrentBridge] URL encode the first parts of the requests. 2018-04-26 22:57:18 +01:00
Nicolas Delsaux
e60b5ab193 Mise à jour du bridge pour WorldOfTanks (#527)
* Mise à jour de l'un de mes bridges fétiches
2018-04-22 12:58:07 +02:00
teromene
b0c7a62f74 [index] Bumped version to 2018-04-20 2018-04-20 17:15:25 +02:00
teromene
57b15a089e Added DiscogsBridge. Closes #615 2018-04-20 16:57:09 +02:00
teromene
4b7fbe4188 DansTonChatBridge: test before accessing plaintext 2018-04-19 21:00:18 +02:00
Teromene
2390fb58b3 Merge pull request #673 from GregThib/patch-1
DansTonChatBridge: Update to follow DTC website changes
2018-04-19 20:58:01 +02:00
Teromene
1e8d29f6ec Merge pull request #672 from em92/patch-3
[YoutubeBridge] Removed duration in titles on search mode
2018-04-19 20:56:34 +02:00
Eugene Molotov
644d13686c [YoutubeBridge] Removed duration in titles on search mode 2018-04-19 09:03:29 +05:00
teromene
aa0ff1c9b1 Added YGGTorrentBridge. 2018-04-18 21:57:27 +02:00
teromene
539d9f1f06 Add SupInfoBridge, fixes #668 2018-04-18 12:39:45 +02:00
teromene
5ece801ce7 Fix h* display size in HtmlFormat, and fix images being wider than the page. 2018-04-18 12:29:22 +02:00
GregThib
4dcea6d9c9 Update to follow DTC website changes
Now, entry title is optionnal and may be found in h3 HTML element.
Entry content is mandatory and may be found in div[class="item-content"] HTML element.

Moreover, the title may contain simple quotes (here, encoded) so the bridge have to decode first to apply format library function. In case we don't do that, the format function double encode the quote and something like ' could appear.
2018-04-18 12:00:00 +02:00
teromene
d69e2521f1 Removed T411 bridge. Website was closed nearly one year ago. 2018-04-18 11:44:54 +02:00
teromene
7927d73719 Rewrote DemonoidBridge. Fixes #626. 2018-04-17 15:25:02 +02:00
teromene
0620f30ae0 Changed the API key used for SoundCloud bridge. Should fix #599 2018-04-17 14:24:00 +02:00
teromene
795494cfce Added enclosures to InstagramBridge. 2018-04-16 19:34:21 +02:00
teromene
ba8542156c Remove usage of function file_get_contents. 2018-04-16 19:27:20 +02:00
Eugene Molotov
55f112e034 [VkBridge] Rewrited bridge code (#667)
* [VkBridge] Convert special HTML entities to characters in pageName

* [VkBridge] Generate feed item title

* [VkBridge] Remove double backslashes in feed item link

* [VkBridge] Unpin post if pinned

* [VkBridge] Mark reposted messages

* [VkBridge] Correct external link parsing

* [VkBridge] Added article parsing

* [VkBridge] Added video parsing

* [VkBridge] Added photo parsing

* [VkBridge] Added album link parsing

* [VkBridge] Added one more external link selector

* [VkBridge] Using array of link selectors to remove

* [VkBridge] Added document parsing

* [VkBridge] Added sign parsing

* [VkBridge] Fixed incorrect sorting with pinned item

* [VkBridge] More methods to parse documents

* [VkBridge] Save fallback if page name element not found

* [VkBridge] Using post signed as feed item author

* [VkBridge] Fixed document link

* [VkBridge] Coding policy fixes
2018-04-16 10:55:31 +01:00
Mitsukarenai
208fff801d [FDroid] minor fixes for Travis CI 2018-04-15 13:21:48 +02:00
Mitsukarenai
3c9860de43 [FDroid] new bridge 2018-04-15 13:13:10 +02:00
Adam Tygart
a16ec196c5 [NotAlways] Add a bridge for the NotAlways family of sites (#537)
NotAlways right found it necessary to remove their RSS feeds recently. This is a *simple* bridge to grab the ones on the front page. It allows you to filter the articles based on their classification (right, working, romantic, related, learning, friendly, hopeless, unfiltered, or all).
2018-04-15 12:02:37 +01:00
teromene
887fc7b037 Fix GoComics, website completely changed. Fixes #663 2018-04-14 18:15:44 +01:00
teromene
1bd4a40f71 Added GNOME Builder configuration to gitignore. 2018-04-14 18:15:13 +01:00
teromene
494169f959 Added bridge for Pixiv.
This bridge is slow, as caching of images is required (REFERER header required to access the full size images)
2018-04-14 16:19:35 +01:00
logmanoriginal
178177e787 [index] Push version to 2018-04-06 2018-04-06 22:45:33 +02:00
logmanoriginal
1cb83ccea3 [IPBBridge] Use limit for the number of items
The limit was used to specify the number of pages to return from a given
topic which resulted in the number of returned items variing between one
and however many entries are listed on one page.

This commit changes the implementation for the limit to keep loading more
pages until the specified limit is reached. Excessive elements are removed
in order to return the exact amount of items specified by the limit.

This behavior is closer to how other bridges are implemented and makes it
more natural to use without being too confusing. Existing queries must be
updated to account for the new limit.

References #657
2018-04-06 22:25:49 +02:00
sysadminstory
c899399569 [DealabsBridge] Follow the website changes (#660) 2018-04-06 21:25:41 +02:00
LogMANOriginal
0f93370e92 Merge pull request #654 from LogMANOriginal/cURL
Use cURL instead of file_get_contents
2018-04-06 20:49:58 +02:00
logmanoriginal
45c3dcb636 [VkBridge] Simplify header specification 2018-04-06 20:42:19 +02:00
logmanoriginal
ecfc220b10 [KernelBugTrackerBridge] Fix too many parameters requesting HTML DOM 2018-04-06 20:42:19 +02:00
logmanoriginal
4b3efed7ec [YoutubeBridge] Fix too many parameters when using HTML mode 2018-04-06 20:42:19 +02:00
logmanoriginal
bc28c5da8e [contents] Set CURLOPT_HTTPHEADER only if the provided array contains data 2018-04-06 20:42:19 +02:00
logmanoriginal
5bd9c1611d [contents] Limit cURL protocols to HTTP and HTTPS 2018-04-06 20:42:19 +02:00
logmanoriginal
6caca4946b bridges: Fix bridges with custom headers and options
This commit fixes bridges which called getContents, getSimpleHTMLDOM
or getSimpleHTMLDOMCached with custom settings.
2018-04-06 20:42:19 +02:00
logmanoriginal
ee78e7613f [contents] Replace file_get_contents by cURL
cURL is a powerful library specifically designed to connect to many
different types of servers with different types of protocols. For
more detailed information refer to the PHP cURL manual:

- http://php.net/manual/en/book.curl.php

Due to this change some parameters for the getContents function were
necessary (also applies to getSimpleHTMLDOM and getSimpleHTMLDOMCached):

> $use_include_path removed

  This parameter has never been used and doesn't even make sense in
  this context; If set to true file_get_contents would also search
  for files in the include_path (specified in php.ini).

> $context replaced by $header and $opts

  The $context parameter allowed for customization of the request in
  order to change how file_get_contents would acquire the data (i.e.
  using POST instead of GET, sending custom header, etc...)

  cURL also provides facilities to specify custom headers and change
  how it communicates to severs. cURL, however, is much more advanced.

  - $header is an optional parameter (empty by default). It receives
    an array of strings to send in the HTTP request header.

    See 'CURLOPT_HTTPHEADER':

    "An array of HTTP header fields to set, in the format
    array('Content-type: text/plain', 'Content-length: 100')"

    - php.net/manual/en/function.curl-setopt.php

  - $opts is an optional parameter (empty by default). It receives
    an array of options, where each option is a key-value-pair of
    a cURL option (CURLOPT_*) and it's associated parameter. This
    parameter accepts any of the CURLOPT_* settings.

    Example (sending POST instead of GET):

    $opts = array(
      CURLOPT_POST => 1,
      CURLOPT_POSTFIELDS => '&action=none'
    );
    $html = getContents($url, array(), $opts);

    Refer to the cURL setopt manual for more information:
    - php.net/manual/en/function.curl-setopt.php

> $offset and $maxlen removed

  These options were supported by file_get_contents, but there doesn't
  seem to be an equivalent in cURL. Since no caller uses them they are
  safe to remove.

Compressed data / Encoding

  By using cURL instead of file_get_contents RSS-Bridge no longer has
  to handle compressed data manually.

  See 'CURLOPT_ENCODING':

  "[...] Supported encodings are "identity", "deflate", and "gzip".
  If an empty string, "", is set, a header containing all supported
  encoding types is sent."

  - http://php.net/manual/en/function.curl-setopt.php

  Notice: By default all encoding types are accepted (""). This can
  be changed by setting a custom option via $opts.

    Example:

    $opts = array(CURLOPT_ENCODING => 'gzip');
    $html = getContents($url, array(), $opts);

Proxy

The proxy implementation should still work, but there doesn't seem
to be an equivalent for 'request_fulluri = true'. To my understanding
this isn't an issue because cURL knows how to handle proxy communication.
2018-04-06 20:42:19 +02:00
logmanoriginal
2df2623430 [index] Add 'curl' extension check 2018-04-06 20:42:19 +02:00
logmanoriginal
de5f850cdb [index] Fix indentation using tabs 2018-04-06 20:34:44 +02:00
teromene
ac6847045c Catch Errors in order to display a message in more cases. We also catch Exceptions to maintain compat with php 5.
Add check for simplexml extension.
2018-04-04 19:02:40 +01:00
logmanoriginal
df6da837dc [FacebookBridge] Return error if username starts with slash
Requesting a username with a leading slash would cause error 500
because the requested URI would contain two slashes in a row.

For example username "/test" would result in:
https://facebook.com//test

References #628
2018-03-23 21:23:30 +01:00
Eugene Molotov
41b7984a4e [YoutubeBridge] Playlist mode: faster feed generating if item count is less or equal to 15 (#648)
* [YoutubeBridge] Playlist mode: faster feed generating if item count is less or equal to 15
2018-03-19 12:41:52 +00:00
teromene
38c7e0272e Add hashtag support to InstagramBridge.
Fixes  #629
2018-03-19 12:29:24 +00:00
teromene
29c690dbcd Fix InstagramBridge, thanks to @pintassilgo comments.
Fixes #646
2018-03-19 12:17:42 +00:00
LogMANOriginal
8ba817478b Implement customizable cache timeout (#641)
* [BridgeAbstract] Implement customizable cache timeout

The customizable cache timeout is used instead of the default cache
timeout (CACHE_TIMEOUT) if specified by the caller.

* [index] Add new global parameter '_cache_timeout'

The _cache_timeout parameter is an optional parameter that can be
used to specify a custom cache timeout. This option is enabled by
default.

It can be disabled using the named constant 'CUSTOM_CACHE_TIMEOUT'
which supports two states:

> true: Enabled (default)
> false: Disabled

* [BridgeAbstract] Change scope of 'getCacheTimeout' to public

* [html] Add cache timeout parameter to all bridges

The timeout parameter only shows if CUSTOM_CACHE_TIMEOUT has been set
to true. The default value is automatically set to the value specified
in the bridge.

* [index] Disable custom cache timeout by default
2018-03-14 18:06:36 +01:00
Eugene Molotov
cacbe90102 [YoutubeBridge] Sort playlist items by publication date (#643) 2018-03-13 11:24:40 +00:00
Antoine Cadoret
cb91cd5d2f Fix SteamBridge (#637) (#639)
Fixes #639
2018-03-12 09:22:34 +00:00
sysadminstory
52dfa3fe76 [RadioMelodieBridge] Add new bridge (#640) 2018-03-11 15:38:07 +01:00
logmanoriginal
29a1c7ac09 [index.php] Add extension check for 'mbstring'
The mbstring extension is required by all formats in order to convert multi-
byte characters to UTF-8. This commit adds an extension check to throw an
error message if the extension is not enabled.
2018-03-07 19:11:47 +01:00
teromene
6eea51eeeb Fix SteamBridge.
Fixes #636
2018-03-07 10:24:33 +00:00
teromene
2149af0e74 Fix Pinterest bridge, remove the old JSON parsing, and return original sized image.
Fixes #632
2018-03-06 12:01:48 +00:00
teromene
142a647b7a Merge branch 'master' of github.com:RSS-Bridge/rss-bridge 2018-03-06 11:27:37 +00:00
teromene
6e916ddd35 Fix Arte7Bridge.
Fixes #633
2018-03-06 11:26:16 +00:00
Eugene Molotov
159b00145d [VkBridge] Setting feed title (#635)
* [VkBridge] Setting feed title
2018-03-05 09:46:15 +00:00
Mitsukarenai
26ce16baa2 [PlanetLibre] remove bridge (origin now has RSS) 2018-03-03 21:04:40 +01:00
sysadminstory
0622fe142b Dealabs : Added Groupes Feeds and Feed name is set according to parameters (#630)
* [DealabsBride] Added Groupes Feeds
2018-03-01 17:10:34 +00:00
logmanoriginal
4805b52d42 [YoutubeBridge] Fix typo 2018-02-16 22:35:00 +01:00
logmanoriginal
962617086e [YoutubeBridge] Remove superfluous div selectors 2018-02-16 22:31:47 +01:00
logmanoriginal
4f6277b6b5 [YoutubeBridge] Fix parsing author name breaks the bridge
The author name is parsed by searching a string within the entire
HTML document:

$author = $html->innertext;
$author = substr($author, strpos($author, '"author=') + 8);
$author = substr($author, 0, strpos($author, '\u0026'));

This solution will return big portions of the HTML document if
the strpos function returns zero (not found).

This commit replaces the previous implementation by searching for
a specific script tag and making use of the JSON data inside it.

References #580
2018-02-16 22:31:29 +01:00
logmanoriginal
5aaab9eb8c [YoutubeBridge] Skip unavailable videos 2018-02-16 22:11:03 +01:00
sysadminstory
ef402bb5c3 [DealabsBride] Fix for the new site (#595)
* [DealabsBride] Fix for the new site
2018-02-14 11:03:44 +00:00
LogMANOriginal
85ac9001d6 [IPBBridge] Add bridge (#564)
This bridge returns feeds for any URI that is compatible with the
IPB implementation (currently 4.x). Older versions might work, but
there is no guarantee.

Only forum and topic URIs are supported!

The bridge automatically checks if natural feeds are available (by
adding '.xml' to the URI). If so the feed is returned. Otherwise
the bridge will attempt to identify the content type and build a
feed accordingly.

Valid URIs are forums and topics. For forums the first page is
returned, for topics the last one. Elements are ordered such that
the latest entry is returned first (oldest-to-newest)

The optional parameter '&limit=' specifies how many pages should
be loaded (default: 1). Topics are loaded in reverse order.
=> Does not work with forums!

Images are provided as enclosures and scaled to a max-size of
400x400 pixels by default (Except for natural feeds).

The content is filtered before being returned:
- Unnecessary tags are removed (iframes, etc...)
- Styles for blockquotes are restored (grey background)

Closes #507
2018-02-13 21:46:33 +01:00
Mitsukarenai
7939bffcdd fix: TébéoBridge Travis cleanup 2018-02-11 19:08:19 +01:00
Mitsukarenai
bb58aa8e31 New bridge: Tébéo 2018-02-11 16:56:34 +01:00
Ruslan
1d35149191 Update VkBridge (#625) 2018-01-30 16:57:07 +00:00
Tameroski
be03764029 Fixing double quote issue at the end of URL (#623) 2018-01-23 11:27:45 +00:00
Matt DeMoss
a07874d468 Initial commit for Bloomberg bridge with top stories and search (#607)
* initial commit for Bloomberg bridge with top stories and search
2018-01-12 12:08:15 +00:00
Matt DeMoss
90d7ae8776 Fix twitter list filter test #613, fix and change getName() for lists. (#614) 2018-01-12 12:07:40 +00:00
Teromene
93e0562353 Merge pull request #610 from mdemoss/YouTubeTitle-#609
You tube title fix for #609
2018-01-11 12:09:38 +00:00
Teromene
4c5d547d9c Merge pull request #608 from mdemoss/PcGamerBridge
Pc gamer bridge
2018-01-11 12:08:10 +00:00
Teromene
9a3a64010f Merge pull request #620 from RSS-Bridge/teromene-patch-2
Update MixCloudBridge.php
2018-01-11 11:48:29 +00:00
Teromene
e59a6f4c9e Update MixCloudBridge.php
Fix whitespace at start of line
2018-01-11 11:44:51 +00:00
Teromene
1506e68587 Merge pull request #619 from RSS-Bridge/teromene-patch-1
Update .travis.yml
2018-01-11 11:44:37 +00:00
Teromene
671cba4f68 Update .travis.yml
Try to fix build failure
2018-01-11 11:41:25 +00:00
Teromene
374eb8f4bf Merge pull request #617 from adamchainz/patch-1
README - sort lists alphabetically
2018-01-10 14:05:05 +00:00
Adam Johnson
60f7a2b3e4 README - sort lists alphabetically
This makes them easier to scan and check "does rss-bridge support service X I'm interested in?" :)
2018-01-10 11:45:55 +00:00
Teromene
7744172c63 Merge pull request #616 from lalannev/patch-1
Update LegifranceJOBridge.php
2018-01-09 17:19:28 +00:00
lalannev
5a763aee8d Update LegifranceJOBridge.php 2018-01-09 14:57:17 +01:00
Matt DeMoss
c14b2c6905 address phpcs style errors 2017-12-28 20:20:24 -05:00
Matt DeMoss
0871376922 store feed name in new variable, switch getName on queriedContext, remove 'bridge' from name for feeds, fixes #609 2017-12-28 20:20:24 -05:00
Matt DeMoss
c5fe9a6dc0 mark places where a new variable is needed 2017-12-28 20:20:24 -05:00
Matt DeMoss
fbbcd02384 apply phpcbf for automatic style fixes 2017-12-24 16:45:56 -05:00
Matt DeMoss
d34987f9c1 PC Gamer bridge initial commit with most read stories 2017-12-24 16:40:59 -05:00
Teromene
9e0565c655 Merge pull request #604 from TwizzyDizzy/master
Fix double forward-slash in returned post URI leading to 404
2017-12-14 16:43:33 +00:00
Thomas Dalichow
443081c90b Fix double forward-slash in returned post URI leading to 404 2017-12-06 22:17:46 +01:00
Teromene
03fc09e3c6 Merge pull request #602 from TwizzyDizzy/master
Fake user agent as Mixcloud blocks certain User-Agents
2017-12-01 17:29:26 +00:00
Thomas Dalichow
45323c2b2f Fake user agent as Mixcloud blocks certain User-Agents 2017-12-01 17:28:57 +01:00
Teromene
67ee73782c Merge pull request #582 from sysadminstory/master
[DealabsBridge] Add new bridge
2017-10-18 10:53:46 +01:00
sysadminstory
2bb9a29ddc Delete usefull whitespace 2017-10-17 23:37:09 +02:00
sysadminstory
5cbd363597 Coding style fix
Fixed the bridge to follow the project coding style
2017-10-17 23:30:27 +02:00
Teromene
aa6ded0ea4 Merge pull request #593 from b1nj/master
Update saison AllocineFRBridge
2017-10-17 19:02:54 +01:00
sysadminstory
3c61dc2b57 Merge remote-tracking branch 'upstream/master' 2017-10-17 14:53:22 +02:00
B1nj
3e528ddccf Update saisons AllocineFRBridge 2017-10-16 22:24:49 -04:00
teromene
cba65d6d08 [Arte7Bridge] Fix Arte7 bridge, use the API 2017-10-12 18:12:31 +01:00
Teromene
8d418611a2 Merge pull request #589 from mickael-bertrand/patch-2
Updater torrent9 URI
2017-10-12 17:18:04 +01:00
teromene
98b0f0f8ba [Core] Verify the presence of the array keys before accessing them.
Fixes  #588
2017-10-12 17:14:34 +01:00
Teromene
6f66e6d9be Merge pull request #592 from ldidry/fix-gocomics
Update GoComicsBridge
2017-10-11 11:34:37 +01:00
Luc Didry
8b06299bad Update GoComicsBridge 2017-10-11 10:03:29 +02:00
MickaëlBERTRAND
5a99981827 Updater torrent9 URI 2017-10-08 19:21:10 +02:00
Teromene
e30ad3feb4 Add support for running rss-bridge from the CLI 2017-09-25 19:14:02 +02:00
logmanoriginal
77657a9154 [.travis.yml] Refactor script 2017-09-24 18:45:58 +02:00
logmanoriginal
3059b1ea80 [YoutubeBridge] Skip Ads
The search might return unrelated videos (Ads) that are inserted
between regular search results. This adds a check to skip Ads.

Closes #571
2017-09-24 17:25:47 +02:00
LogMANOriginal
4037c34393 [TwitterBridge] Add category for lists (#545)
This adds a new option to generate feeds from Twitter lists using
an optional filter (string comparison).
2017-09-24 16:59:45 +02:00
logmanoriginal
e671a2ad02 [.travis.yml] Fix configuration to work with Ubuntu Trusty
Travis CI upgraded the linux build environment from Ubuntu Precise
to Ubuntu Trusty with Trusty becoming the default build environment
as of August 2017:

  https://docs.travis-ci.com/user/reference/overview/

A bug in the configuration of the Ubuntu Trusty distro causes all
builds except nightly to fail. The PHP include_path is set to

  include_path='.:/home/travis/.phpenv/versions/5.6.31/share/pear'

instead of

  include_path='.:/home/travis/.phpenv/versions/5.6.31/lib/php/pear'

which causes phpcs to fail because it cannot resolve import paths.

This commit adds a hotfix to .travis.yml that circumvents the
issue by overwriting the include_path during initialization. This
hotfix should be removed once a solution is found.

This bug is tracked via
https://github.com/travis-ci/travis-ci/issues/8487
2017-09-23 20:48:28 +02:00
logmanoriginal
1ea091f215 [.travis.yml] Fix Tavis CI build error
Travis CI recently updated the default distribution from Ubuntu
Precise to Ubuntu Trusty, which causes all builds except nightly
to fail.

For unknown reasons phpcs is unable to locate "PHP/CodeSniffer/
autoload.php" causing it to fail with a fatal error

The root cause of the failure is unknown. We explicitly return to
the previous build system (Ubuntu Precise) for builds to work again.

See the migration guide for reference:
https://docs.travis-ci.com/user/precise-to-trusty-migration-guide/

Notice: Ubuntu Precise is retired as of September 2017 and will be
decommissioned in the near future:
https://docs.travis-ci.com/user/reference/overview/
2017-09-23 18:38:47 +02:00
logmanoriginal
87fa4ae3ac [.travis.yml] Fix warning channel pear.php.net has updated its protocol 2017-09-23 18:05:09 +02:00
sysadminstory
d7a1dca004 [DealabsBridge] Conform to coding policy
- If no there are no results, an explicit message is now returned
- Commas are now following the coding policy
- Lines are no longer more than 80 chars when possible
2017-09-19 02:08:22 +02:00
sysadminstory
fe48340327 [DealabsBridge] Add new bridge 2017-09-05 21:03:21 +02:00
logmanoriginal
b4c6aa41a7 [index] Return error if no format is specified when requesting a bridge 2017-08-28 20:45:06 +02:00
metaMMA
1696aee212 [DemonoidBridge] Add new bridge 2017-08-28 20:00:52 +02:00
metaMMA
585379d47a [ThePirateBayBridge] Add instructions
Added additional instructions for: 'username search' and 'category
search' next to instructions for 'keyword search'.

Changed variable name from underscore to camelCase.
2017-08-28 20:00:00 +02:00
logmanoriginal
2595b5d7d8 [index] Bump version 2017-08-19 21:09:48 +02:00
logmanoriginal
f858adc884 [CHANGELOG] Add 2017-08-19 2017-08-19 21:08:44 +02:00
logmanoriginal
44e135ce1e [CHANGELOG] Fix layout 2017-08-19 20:12:17 +02:00
logmanoriginal
9a9ce30b16 [YoutubeBridge] Fix issues loading playlists
Videos that are part of a playlist have the playlist ID encoded in
the URI. When loading the video info the page contents change unex-
pectedly due to the playlist being part of the page.

This removes any trailing parameters from the video ID in order to
ensure only pure videos are loaded at all times.
2017-08-19 18:51:30 +02:00
logmanoriginal
0e2b80d5d7 [YoutubeBridge] Fix error on certain keywords
References #569
2017-08-17 19:26:04 +02:00
logmanoriginal
1b1ab6a66e [validation] Fix error on undefined optional numeric value
Providing no value for an optional numeric parameter results in
error "Parameter *** is invalid!"

This is caused by the validation function ignoring the 'required'
attribute when loading and checking input parameters.

This commit adds checks to determine whether the 'required' attri-
bute is defined and active before returning the error message.

References #570:
2017-08-17 19:02:50 +02:00
mcbyte-it
0284e9d488 [GoComicsBridge] Fix for page structure changes (#568)
GoComics changed comic page structure, so this patch fixes it

Closes #565
2017-08-17 18:35:41 +02:00
logmanoriginal
f91309c7e4 [index] Use constant WHITELIST_FILE all the way 2017-08-12 19:15:16 +02:00
logmanoriginal
cd012e115b [index] Show bridge options when loading with URL fragment
Loading the page with an URL fragement (#bridge-*) should result in
the bridge showing all parameters by default. Unfortunately this is
not possible using PHP, which is why a new JavaScript function is
needed (select.js)

That way, when returning from a bridge ('back to rss-bridge') will
keep the selected bridge active (only works for HTML format).
2017-08-11 19:39:16 +02:00
logmanoriginal
df9e3968dc [index] Add GET parameter 'q' for search queries 2017-08-11 17:43:15 +02:00
logmanoriginal
c237eaa254 [style] Fix All input boxes are center aligned
f757d7d1a5 introduced a bug where all
text input boxes were centered instead of just the search bar.

In order for this to work properly the global styles must be applied
before specific styles for the search bar.
2017-08-10 20:27:27 +02:00
logmanoriginal
f757d7d1a5 [style] Center search cursor and hide placeholder
The search bar doesn't feel right if the placeholder is centered,
while the text cursor is left-aligned. The cursor should appear
instead of the placeholder (at the same position).

Added styles to center the text cursor and hide the placeholder
when selecting the input field.

Tested in:
 - Firefox 54 & 55
 - Chromium 60 (compatible with Chrome 60)
 - Microsoft Edge (partially working!)

--- Microsoft Edge ---

Due to a bug in the Microsoft Edge browser, the text cursor is not
centered as long as the placeholder is defined (which it is always)

More information:
  https://stackoverflow.com/a/33224868

Official bug report:
  https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/4468563/

----------------------
2017-08-10 14:35:09 +02:00
logmanoriginal
4fb1366aaf [FeedExpander] Fix Serialization of 'SimpleXMLElement' is not allowed 2017-08-10 13:35:19 +02:00
logmanoriginal
8166e33e7f [FeedExpander] Remove whitespace from source content
Whitespace at the beginning of feeds causes parsing errors. This is
an example using an ill-formatted RSS feed:

   "XML or text declaration not at start of entity"
-- https://validator.w3.org

This commit automatically removes all proceeding and trailing white-
space from the source content before resume parsing.
2017-08-10 13:20:35 +02:00
Quentin de Longraye
ff3b1c9eb2 [DribbbleBridge] Add dribble bridge listing last dribble popular shots (#558) 2017-08-06 20:29:21 +02:00
logmanoriginal
4924769549 [validation] Remove superfluous if-statement 2017-08-06 13:45:24 +02:00
logmanoriginal
e4fa963bdf [validation] Return null on invalid number 2017-08-06 13:43:23 +02:00
logmanoriginal
54e8bb2228 [VineBridge] Remove bridge
On Oct 27, 2016 the discontinuation of Vine was announced:
https://medium.com/@vine/important-news-about-vine-909c5f4ae7a7

"Today, we are sharing the news that in the coming months we’ll be
discontinuing the mobile app."

https://vine.co/ is still online, but has been put into an archive
indefinitely. As the site does not allow further uploads, this
bridge serves no further purpose.
2017-08-06 13:03:10 +02:00
logmanoriginal
99e7e7876e exception: Use built-in HTTP response codes
PHP >= 5.4 provides a built-in function to generate valid HTTP
error header including the error description: http_response_code()

See: http://php.net/manual/en/function.http-response-code.php
See also: https://stackoverflow.com/a/12018482

This commit removes the '\Http' utility class and replaces all
calls to 'Http::getMessageForCode()' by 'http_response_code()'
2017-08-06 12:55:11 +02:00
logmanoriginal
62c190d841 [Bridge] Remove superfuous variables and statements 2017-08-06 00:04:07 +02:00
logmanoriginal
84d2c02a09 whitelist: Do case-insensitive whitelist matching
Matching whitelisted bridges using a case-insensitive match makes
sense for following reasons:

- Wrong upper/lower case spelling in the whitelist is not easily
discovered. Example: Misspelling 'Youtube' as 'YouTube' will not
show the 'Youtube' bridge (while it is expected to show)

- Two bridges with the same name but different letter casing are
discouraged to prevent confusion and keep the project compatible
with Windows machines
2017-08-06 00:01:32 +02:00
logmanoriginal
fc0ae42450 [GelbooruBridge] Fix bridge not getting tags correctly
Tags are embedded in the 'title' attribute instead of 'alt' as
defined by the ancestor (DanbooruBridge).

The 'title' attribute also contains statistics data ('score:...',
'rating:...') that is now filtered by a custom implementation of
the 'getTags' function (elements that contain a colon are removed)

Closes #560
2017-08-05 22:38:24 +02:00
logmanoriginal
9599f921a5 [DanbooruBridge] Allow descendant classes to override tag collection
Add protected function 'getTags' that receives the current element
and returns a string containing all tags.

References #560
2017-08-05 22:36:14 +02:00
logmanoriginal
e125e9aba1 [LeBonCoinBridge] Fix bridge is marked executable
Closes #561
2017-08-05 22:00:58 +02:00
Pierre Mazière
55a77c734d [LWNprevBridge] Fix everchanging url
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>

Closes #563
2017-08-05 15:56:35 +02:00
logmanoriginal
ccd8af09b9 [index] Use single quotes instead of double quotes 2017-08-05 15:46:16 +02:00
logmanoriginal
f2d02a4187 [index] Simplify debug mode detection
This removes superfluous variables and if-statements when checking
whether the debug mode is active or not.
2017-08-05 15:43:48 +02:00
logmanoriginal
f19d34a5a1 [index] Check permissions for cache folder and whitelist file
* The cache folder requires write permissions at all times
* The whitelist file requires write permissions if it does not
exist (can be created manually)
2017-08-05 15:23:30 +02:00
logmanoriginal
f1534c91e2 [index] Use constant instead of variable for the whitelist file path
Like the cache folder the whitelist file is assumed static and thus
should be defined as constant.
2017-08-05 15:23:08 +02:00
logmanoriginal
cbda060b86 [FacebookBridge] Fix &amp; in URLs
All formats except HTML return &amp; instead of & in URLs causing
all links with parameters (...&id=...) to break.

Facebook does not return valid HTML URIs but instead provides them
with all special characters encoded (like using htmlspecialchars).
This seems to be related to the page being build almost entirely of
script blocks.

This commit adds htmlspecialchars_decode() to URI and content to
reverse the encoding.

References #550
2017-08-04 21:12:48 +02:00
logmanoriginal
f7265ca77b [index] Bump version number 2017-08-03 20:51:59 +02:00
logmanoriginal
629a4c4481 [CHANGELOG] Add 2017-08-03 2017-08-03 20:49:59 +02:00
logmanoriginal
950ae2cc05 [CHANGELOG] Change order of appearance 2017-08-03 20:49:39 +02:00
Pierre Mazière
873a91259f [LWNprevBridge] full rewrite
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
[logmanoriginal@users.noreply.github.com: Fix coding style]
2017-08-03 19:39:50 +02:00
logmanoriginal
c7ec50373a Merge branch 'TwitterWithoutPromotedTweets'
Closes #556
2017-08-03 18:26:06 +02:00
logmanoriginal
c986ff9116 [TwitterBridge] Fix coding style 2017-08-03 17:56:39 +02:00
Pierre Mazière
485b465a24 [TwitterBridge] ignore promoted tweets
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2017-08-03 00:44:21 +02:00
logmanoriginal
a4b9611e66 [phpcs] Add missing rules
- Do not add spaces after opening or before closing parenthesis

  // Wrong
  if( !is_null($var) ) {
    ...
  }

  // Right
  if(!is_null($var)) {
    ...
  }

- Add space after closing parenthesis

  // Wrong
  if(true){
    ...
  }

  // Right
  if(true) {
    ...
  }

- Add body into new line
- Close body in new line

  // Wrong
  if(true) { ... }

  // Right
  if(true) {
    ...
  }

Notice: Spaces after keywords are not detected:

  // Wrong (not detected)
  // -> space after 'if' and missing space after 'else'
  if (true) {
    ...
  } else{
    ...
  }

  // Right
  if(true) {
    ...
  } else {
    ...
  }
2017-07-29 19:55:12 +02:00
LogMANOriginal
38b56bf23a [index] Improve error handling (#555)
Add additional information to error message:

- Name of the bridge
- Possible solutions
- Error description
- Error code
- Error message

* Output type changed from 'text' to 'html'
* Added styles for the error page
* Added a button to remotely open a GitHub issue

Closes #525
2017-07-29 19:16:16 +02:00
logmanoriginal
6e4bc341b7 [FacebookBridge] Replace 'novideo' with 'media_type'
This replaces the 'novideo' parameter with 'media_type' in order
to filter for specific content types. Currently supported:

- 'all': Returns all posts (default)
- 'video': Returns only posts including videos
- 'novideo': Returns only posts that don't include videos

References #553
2017-07-25 16:04:21 +02:00
logmanoriginal
fa2df09b1b [FacebookBridge] Add option to hide posts with facebook videos
This adds a new option 'novideo' that can be set to 'on' or 'off'
in order to skip posts that include facebook videos (does not work
for linked videos like YouTube). This option is 'off' by default.

References #533
2017-07-25 15:41:05 +02:00
logmanoriginal
7dda088b3f [InstagramBridge] Add option to filter for videos and pictures
Adds a new option 'media_type' to select from three choices:

- 'all' (Both): Returns pictures and videos (default choice)
- 'picture': Returns only pictures
- 'video': Returns only videos

References #553
2017-07-25 15:14:37 +02:00
logmanoriginal
f6f3a213ef [DanbooruBridge] Fix broken URI
This fixes broken URIs in the output data caused by duplicate domain
names caused by sites (descendant class Delbooru) providing absolute
URIs instead of relative ones.

References #552
2017-07-25 14:43:29 +02:00
Antoine Cadoret
1faa91ef0f Add SteamBridge (#543) 2017-07-17 15:45:58 +02:00
Corentin Garcia
5caca62677 Update RainbowSixSiegeBridge (#548)
* Fix non-working RainbowSixSiegeBridge

* Updated RainbowSixSiegeBridge to use API to fetch articles

* Fix RainbowSixSiegeBridge coding style

* RainbowSixSiegeBridge fix url coding style error
2017-07-14 22:05:51 +02:00
logmanoriginal
d7ff8b9ac7 [TwitterBridge] Fix title includes anchors in plaintext format
The title attribute includes tags (anchors) instead of raw text.
While this works fine in a browser, using a raw format like plain-
text or json breaks with expected behavior.

This commit changes the order in which functions are applied. By re-
moving anchors AFTER fixing the title, the final result does not
include tags and the title is still fixed.

This bug was introduced by d81b61ccfa

References: #546
2017-07-05 18:42:03 +02:00
logmanoriginal
ab46af9719 [TwitterBridge] Avoid empty content caused by new login policy
Twitter now requires login to access "Tweets & Replies" which
breaks feeds using the default behavior. Using the "Without
replies" option still works.

This commit makes the "Without replies" option default. That way
existing feeds will return contents again. The parameter can still
be checked but its status has no effect anymore.

Notice: The parameter should not be removed as that would cause
any feed using the parameter to stop working because of "Invalid
parameter"

References #544
2017-07-03 19:53:18 +02:00
LogMANOriginal
06babeb644 Merge pull request #541 from Frenzie/filterbridge_real
[FilterBridge] Initial implementation of basic title permit and block
2017-07-03 19:15:28 +02:00
LogMANOriginal
341010b391 Merge pull request #535 from Frenzie/filterbridge
[FeedExpander] Deal with empty item
2017-07-03 19:13:12 +02:00
Frans de Jonge
995d78fa5a [FilterBridge] Initial implementation of basic title permit and block
See the comment https://github.com/RSS-Bridge/rss-bridge/issues/402#issuecomment-305982306

Split off from https://github.com/RSS-Bridge/rss-bridge/pull/535
2017-06-24 15:11:40 +02:00
Frans de Jonge
781e4f1908 [FeedExpander] Deal with empty item 2017-06-24 15:09:15 +02:00
logmanoriginal
ae59b20c0c [TwitterBridge] Fix double slashes in URI
This fixes double slashes in the feed URI (https://twitter.com//...
instead of https://twitter.com/...)

Reported via #538
2017-06-19 00:19:55 +02:00
logmanoriginal
d81b61ccfa [TwitterBridge] Fix missing spaces
This commit improves readability of tweets by adding spaces before
anchors in the text.

- Hide "invisible" tags which were not rendered hidden because of
missing CSS.
- Fix spacing between anchors

Reported via #539
2017-06-19 00:17:46 +02:00
Teromene
9c78362fd7 Warn when accessing a private page. 2017-06-15 11:51:11 +01:00
Teromene
18c6f0126f Fix FB2 bridge 2017-06-15 11:42:59 +01:00
LogMANOriginal
d5f47efcea Merge pull request #533 from Frenzie/patch-2
[RTBFBridge] Update URI
2017-05-28 20:37:11 +02:00
Frans de Jonge
601f61f063 [RTBFBridge] Update URI
A series URL still looks like: https://www.rtbf.be/auvio/emissions/detail?id=3553

But an individual episode has been changed from https://www.rtbf.be/auvio/emissions/detail?id=2217881 to https://www.rtbf.be/auvio/detail?id=2217881
2017-05-28 20:15:59 +02:00
logmanoriginal
8ed4812e00 [FacebookBridge] Add requester languages to HTTP header
If no accepted languages are specified Facebook will guess your
language. This guess can go horribly wrong if your server does not
provide origin information.

This adds a context header with language information when retrieving
page contents. The accepted languages are read from the list of
accepted languages specified by the web browser of the requester.

References #530
2017-05-07 13:27:37 +02:00
Teromene
f38db4d79e Merge pull request #528 from Jocker666z/patch-1
Update URI of t411 bridge
2017-05-04 12:30:18 +01:00
Jocker666z
88d1068406 update uri 2017-05-04 12:01:14 +02:00
logmanoriginal
627038e2fa [YoutubeBridge] Improve URL handling in video descriptions
This improves the translation of regular text to anchors by adding
support for additional characters '?&=-_' to fix common URLs.

Notice: The regex pattern is by no means complete. That means it is
likely to break in the future. More sophistiated solutions however
are insanely complex. See: http://stackoverflow.com/a/190405

References #520
2017-05-02 22:03:44 +02:00
logmanoriginal
5b541e380a [TwitterBridge] Optimize returned image sizes
Twitter provides an easy way to receive various image sizes based
on the same image URI:

https://dev.twitter.com/overview/api/entities-in-twitter-objects

We support different sizes: thumb, small, medium and large. The
media_url defaults to medium but you can retrieve the media in
different sizes by appending a colon + the size key (for example:
https://pbs.twimg.com/media/A7EiDWcCYAAZT1D.jpg:thumb).

-- Twitter Developer Documentation

TwitterBridge now makes use of this feature in order to provide
thumbnail images in the content and original sized images as
enclosures.

References #526
2017-05-02 21:45:26 +02:00
Mitsu
c375ddd6ab Merge pull request #521 from Frenzie/master
WikipediaBridge: fix French Wikipedia
2017-04-28 19:26:54 +02:00
Frans de Jonge
44c3110db0 WikipediaBridge: fix French Wikipedia 2017-04-28 19:15:23 +02:00
logmanoriginal
120e74c1b4 [YoutubeBridge] Improve readability of feed contents
Previously feed contents were rendered as one block of text with
no structure. This brings back the structure of original video
descriptions and makes links in the description work again.

References #520
2017-04-27 21:44:26 +02:00
LogMANOriginal
890ba69116 Merge pull request #518 from rogerdc/master
Adding DiceBridge.php
2017-04-25 21:59:57 +02:00
LogMANOriginal
d6da2ce406 Merge pull request #519 from sysadminstory/master
[AllocineFRBridge] Update Faux Raccord link
2017-04-25 21:48:48 +02:00
Gilles Maurer
0eb5711a68 [AllocineFRBridge] Update Faux Raccord link
This update the link to last season of the Show "Faux raccord"
2017-04-25 01:11:50 +02:00
rogerdc
a4ef42c2e9 Adding DiceBridge.php
Dice.com is a technology-oriented job search site. This bridge allows you to create RSS feeds for the jobs listed there.

Happy job hunting!
2017-04-23 19:19:50 -05:00
logmanoriginal
28331e7cd6 [BridgeAbstract] Return cached infos when using cached items
Re-requesting the same feed in normal mode (not debug mode)
returns the cached version. The name and URI of the feed
however was not returned.

This adds checks to getName() and getURI() in order to return
the cached infos when using the cached version.

Notice: For this to work correctly all bridges must call to
parent::getName() and parent::getURI() respectively if the
queriedContext is not defined. Example:

switch($this->queriedContext){
	case 'My context':
		// do your stuff here!
		break;
	default: parent::getName();
}
2017-04-23 21:06:25 +02:00
logmanoriginal
6eadc6ca6f [TwitterBridge] Show quotes and pictures
This adds new features to show quotes and pictures in feeds.

Quotes will show up on top of a tweet and are separated from
the quoting feed by a horizontal line.

Pictures that are embedded in the tweet will be captured and
attached to the feed using enclosures. By default the picture
will also be shown in the feed itself. This can be disabled
using the option '&noimg=on'

Some codes are now split into separate functions so they can be used
for tweets and quotes alike.
2017-04-22 16:01:00 +02:00
logmanoriginal
638d173b70 [PinterestBridge] Fix checkbox not working
Changed behaviour of the checkbox to use the custom parser when
active. That way if the parameter is missing the default value
applies and the feed returns from the provided RSS

Reported via #498
2017-04-10 14:34:45 +02:00
logmanoriginal
a9535797e6 [ShanaprojectBridge] Don't throw error if timestamp is missing 2017-04-10 13:38:02 +02:00
logmanoriginal
fc9084eb17 [MangareaderBridge] Fix double forward slashes
Double forward slashes caused all external links in the feed to break
2017-04-10 13:20:07 +02:00
logmanoriginal
e221358ead [FacebookBridge] Handle summary posts
Previously summary posts were ignored which resulted in the last
two posts not showing up in the feed (the latest two are shown in
the summary post).

Now summary posts are treated like regular posts, returning them
as part of the regular feed.

References #502, #505
2017-04-10 13:04:41 +02:00
logmanoriginal
2500d0df93 [PinterestBridge] Fix implementation after DOM changes
Due to breaking DOM changes this bridge required re-implementation.
With this fix the brige will make use of the JSON data embedded in
the returned HTML. The content returned for all contexts is similar
with only a few differences due to limitations of the JSON.

Feeds returned for a given username and board will by default make
use of the provided RSS feed instead of using the custom filter.
This bahaviour can be changed by setting the  optional parameter
'&r=off' (on by default)

Notice: The JSON data for userdata and search results is very
different, so two functions were implemented to account for that.

References #498
2017-04-09 23:38:35 +02:00
logmanoriginal
4124c707d4 [SexactuBridge] Fix typo 2017-04-09 21:44:05 +02:00
logmanoriginal
8e84b52152 [SexactuBridge] Fix URI and timestamp
* const DOMAIN is not supported, it must be const URI
* strtotime should be used instead of date_parse in order to
receive a valid integer
* Some small readability enhancement
2017-04-09 21:33:50 +02:00
Nicolas Delsaux
f3b6b264d3 [SexactuBridge] Use most modern version of bridge api and cached pages (#504)
Fixed #503 to use most modern version of bridge api and cached pages
2017-04-09 21:15:01 +02:00
logmanoriginal
360f9da072 [EtsyBridge] Add new bridge
This bridge generates feeds for a given search term, optionally
adds the picture to the content and allows for additional query
extensions (GET parameters) to be passed to the bridge. That
way custom filter can be applied without the need to reproduce
them in this bridge (they got a lot!)

Etsy provides a good set of feeds as described here:
https://www.etsy.com/help/article/100

(so there is no need to include them here)

References #492
2017-03-27 20:18:19 +02:00
logmanoriginal
e3b335b9ff [WikiLeaksBridge] Add new bridge
This bridge will fetch contents from https://wikileaks.org

Available options are:

- Category: Defines a list of categories to select from
- Show teaser: Defines whether to show the teaser or not

Notice: Feeds provided by WikiLeaks do not work, see
https://wikileaks.org/wiki/RSS

Closes #489
2017-03-26 17:58:26 +02:00
logmanoriginal
9acd30a5c5 [GooglePlusPostBridge] Autofix user names
User names can either be an ID (series of numbers), or an actual
name, where the name always starts with a '+'.

This commit adds a check for automatically fixing provided user
names which are missing the '+'.
2017-03-26 16:50:42 +02:00
logmanoriginal
3276d4e3d5 [GooglePlusPostBridge] Fix content loading
- Do not force language via HTTP header
The header enforced the language to be french which caused problems parsing
the exact time due to spellings (strtotime cannot work with 'semaines'). If
further issues are experienced try forcing en-us instead.
=> This should really be done in the RSS-Bridge core

- Fix loading problems due to pinned articles
Pinned articles do not provide a timestamp. Building the timestamp step-by-step
solves parsing errors.

- Use class names instead of CSS paths
CSS paths change based on the article. Pinned articles provide a different
DOM structure which caused parsing errors.

Reported via #499
2017-03-26 16:41:20 +02:00
Corentin Garcia
88586381e7 [GithubSearchBridge] Added github search bridge (#500)
* [GithubSearchBridge] Added github search bridge, only repos search atm
2017-03-25 11:24:00 +00:00
niawag
ebe897f120 Create KATBridge.php (#501)
Create KATBridge.php
2017-03-25 11:23:36 +00:00
Teromene
1a4c3f4418 Add a search bar to simplify looking for a bridge. (#494)
* Add a search bar to simplify looking for a bridge.

* Fix phpcs line length.

* Change the phpcs config.
2017-03-21 20:31:10 +00:00
Corentin Garcia
2ac0469750 Updated 4 bridges to use HTTPS (#497)
* [NextInpactBridge] Use https

* [InstagramBridge] Use https

* [GBAtempBridge] Use https

* [LeBonCoinBridge] Use https
2017-03-21 20:27:12 +00:00
Teromene
c0181d8d41 Merge pull request #496 from corenting/patch-2
[RainbowSixSiegeBridge] Added bridge for Rainbow Six Siege blog
2017-03-21 20:26:36 +00:00
Corentin Garcia
ea3073e27f [RainbowSixSiegeBridge] Added bridge for Rainbow Six Siege blog 2017-03-20 21:32:31 +01:00
LogMANOriginal
20ea75994d Merge pull request #495 from corenting/patch-1
[NasaApodBridge] Use HTTPS instead of HTTP
2017-03-20 19:02:32 +01:00
Corentin Garcia
a84c245fa0 [NasaApodBridge] Use HTTPS instead of HTTP 2017-03-20 14:26:08 +01:00
logmanoriginal
b48a44c979 [UsbekEtRicaBridge] Add new bridge
Adds a new bridge to fetch contents from https://usbeketrica.com/
Feeds are build from cards displayed on the front page

This bridge provides two options:
- limit: Defines how many articles are returned
- fullarticle: Defines whether or not the full article is retured

Requested via #457
2017-03-19 14:32:59 +01:00
logmanoriginal
c6ce453c47 [MixCloudBridge] Fix bridge broken after DOM changed
This commit fixes DOM changes reported via #436
New DOM introduced via
https://blog.mixcloud.com/2017/01/10/take-a-look-at-the-new-and-improved-mixcloud/
2017-03-19 12:04:59 +01:00
logmanoriginal
bd92392921 [GooglePlusPostBridge] Fix bridge implementation
This bridge was broken due to DOM changes. This commit fixes
most of the broken code. Hashtags do no longer work because
they are no longer supported/provided.

The timing might be off as the source only provides a rough
relative value like '1 hour' or '1 year'.

Closes #485
2017-03-18 21:09:06 +01:00
Teromene
59025d96bc Add an indicator to show the HTTP status of the site. (#483) 2017-03-18 19:02:18 +00:00
logmanoriginal
155c0ac6f0 Merge branch 'ImproveBridge' of https://github.com/logmanoriginal/rss-bridge 2017-03-17 18:42:37 +01:00
logmanoriginal
596b9143a8 [TwitterBridge] Add option to hide retweets
Requested via #491
2017-03-17 18:41:35 +01:00
logmanoriginal
a2108c784f [FeedExpander] Properly cast simplexml elements
This fixes a possible cause of
"Serialization of 'SimpleXMLElement' is not allowed"
reported via #487
2017-03-13 22:12:11 +01:00
Teromene
c803396d7e Correct phpcs check. 2017-03-09 22:27:14 +00:00
Teromene
ac518ca297 Fix line return in user-agent. 2017-03-08 11:47:55 +00:00
Teromene
1763a1518c Restore the ability to whitelist all the bridges by putting a wildcard into the whitelist file. 2017-03-08 10:56:39 +00:00
Teromene
2dda74dfe7 Add contribution guidelines. 2017-03-07 11:46:54 +00:00
Teromene
b1c2a69102 Fix WebFailBridge (again).
It seems that they blacklist weird user agents, but not immediately. Switched to Firefox user agent, should stop causing problems.
2017-03-03 14:19:10 +00:00
Teromene
bf7ce98719 Fix VKBridge. 2017-03-03 14:14:05 +00:00
Teromene
8b2fdb3937 Add a function to convert the background-image attribute to an actual image. 2017-03-03 14:13:29 +00:00
Teromene
5d41a74067 Add WordPressPluginUpdateBridge.
Fix phpcs check in WebFailBridge.
2017-03-03 13:27:41 +00:00
Teromene
100f3cd56d Fix Webfailbridge, change the user-agent used for the request. 2017-03-03 12:10:23 +00:00
logmanoriginal
8f3c56b184 Merge branch 'ImproveCore' of https://github.com/logmanoriginal/rss-bridge 2017-02-18 13:48:15 +01:00
logmanoriginal
16bdf6b204 links: Rename defaultImageSrcTo to defaultLinkTo
This function not only fixes image sources, but also anchors
2017-02-18 13:41:45 +01:00
logmanoriginal
cf7da1d41c [html] Fix anchors after fixing images
Anchors will be fixed in a similar way as it is done with images,
so it can be done in one go.
2017-02-18 13:40:58 +01:00
logmanoriginal
bb8e7495d8 [html] Fix img src replacement not working
strpos returns false if the needle was not found. See:
http://php.net/manual/en/function.strpos.php#refsect1-function.strpos-returnvalues
2017-02-18 13:13:40 +01:00
logmanoriginal
5de03d6b9f [FileCache] Use serialize instead of json_encode
json_encode causes high memory footprint on large input data,
where serialize is less problematic.

Example: When using AcrimedBridge items contain pictures in
raw format (entire picture) which leads to a file size of about
2MB using serialize. json_encode will allocate about 98MB of
memory for encoding, causing memory exhausion errors (PHP
allows for 128MB of memory by default)
2017-02-18 12:54:26 +01:00
logmanoriginal
1d26c7f1c3 [FileCache] Do not delete .gitkeep
This commit reduces the chance of accidentally removing the cache
folder from repository.
2017-02-18 10:23:50 +01:00
logmanoriginal
790bd17d41 Merge branch 'MoinMoinBridge' of https://github.com/logmanoriginal/rss-bridge 2017-02-18 10:16:05 +01:00
logmanoriginal
1dcef02f27 [MoinMoinBridge] Add new bridge
This bridge returns feeds for each section (via given separator)
from a given MoinMoin compatible wiki.

The separator can be any tag of the following:
- h1
- h2
- h3
- li
- a

The number of items returned can be specified.
For anchor tags (a) the bridge can optionally follow the anchor to
the linked page and return it as content.
2017-02-18 03:13:20 +01:00
logmanoriginal
801ea837c9 Merge branch 'ImproveBridge' of https://github.com/logmanoriginal/rss-bridge 2017-02-17 20:17:57 +01:00
logmanoriginal
9124ed640e [WebfailBridge] Properly handle gifs (DOM changed) 2017-02-17 20:04:38 +01:00
logmanoriginal
6d1e8af982 Merge branch 'ImproveCore' of https://github.com/logmanoriginal/rss-bridge 2017-02-15 19:40:46 +01:00
logmanoriginal
512a4f292b bridges: Return parent::getURI by default 2017-02-15 19:38:32 +01:00
logmanoriginal
c4169f1579 bridges: Return parent::getName by default 2017-02-15 19:38:32 +01:00
logmanoriginal
d93d491d8e core: Use methods to access bridge information
Bridge information were exposed and accessed via public constants
which doesn't work if you want to generate bridges dynamically as
discussed in #402
2017-02-15 19:38:32 +01:00
logmanoriginal
c44fb25845 core: Improve documentation and style for BridgeAbstract and BridgeInterface
Public functions defined in BridgeAbstract also belong to BridgeInterface

getInput may only be used by this class or its children.
2017-02-15 19:36:29 +01:00
LogMANOriginal
761c66d813 Merge pull request #475 from LogMANOriginal/NewCodingStylePolicy
Apply coding style policy
2017-02-15 19:23:25 +01:00
logmanoriginal
ff83410534 style: Fix coding styles 2017-02-14 17:28:07 +01:00
logmanoriginal
d8f5aa3c79 [phpcs] Add sniffs for function declaration and -calls
When declaring a function
- Do not add a space before a comma
- Add a space after a comma
- Add a space after an equal sign

Example:
function myFunction($x, $y, $z = null){...}

When calling a function
- Do not add a space before the opening parenthesis
- Do not add a space after the opening parenthesis
- Do not add a space before the closing parenthesis
- Do not add a space before a comma
- Add a space after a comma

Example:
myFunction('x', 'y', 'z');
2017-02-14 17:03:05 +01:00
logmanoriginal
23430f1c07 [phpcs] Add documentation 2017-02-14 16:50:34 +01:00
logmanoriginal
0c3e58258c [MsnMondeBridge] Fix typo 2017-02-12 16:18:58 +01:00
logmanoriginal
b4f1dc35a1 [FB2Bridge] Split long lines
Splits long lines into short sections without using
string concatenation (.) to prevent errors due to
coding styles.
2017-02-12 15:34:08 +01:00
logmanoriginal
6f24858124 bridges: Fix coding styles
This commit is a squash of all commits that fix coding styles
for the new coding style policy.

[ABCTabsBridge] Fix coding style
[AcrimedBridge] Fix coding style
[AllocineFRBridge] Fix coding style
[AnimeUltimeBridge] Fix coding style
[Arte7Bridge] Fix coding style
[AskfmBridge] Fix coding style
[BandcampBridge] Fix coding style
[BastaBridge] Fix coding style
[BlaguesDeMerdeBridge] Fix coding style
[BooruprojectBridge] Fix coding style
[CADBridge] Fix coding style
[CNETBridge] Fix coding style
[CastorusBridge] Fix coding style
[CollegeDeFranceBridge] Fix coding style
[CommonDreamsBridge] Fix coding style
[CopieDoubleBridge] Fix coding style
[CourrierInternationalBridge] Fix coding style
[CpasbienBridge] Fix coding style
[CryptomeBridge] Fix coding style
[DailymotionBridge] Fix coding style
[DanbooruBridge] Fix coding style
[DansTonChatBridge] Fix coding style
[DauphineLibereBridge] Fix coding style
[DeveloppezDotComBridge] Fix coding style
[DemoBridge] Fix coding style
[DilbertBridge] Fix coding style
[DuckDuckGoBridge] Fix coding style
[DollbooruBridge] Fix coding style
[EliteDangerousGalnetBridge] Fix coding style
[ElsevierBridge] Fix coding style
[EstCeQuonMetEnProdBridge] Fix coding style
[EZTVBridge] Fix coding style
[FacebookBridge] Fix coding style
[FeedExpanderExampleBridge] Fix coding style
[FB2Bridge] Fix coding style
[FierPandaBridge] Fix coding style
[FlickrBridge] Fix coding style
[FootitoBridge] Fix coding style
[FourchanBridge] Fix coding style
[FuturaSciencesBridge] Fix coding style
[GBAtempBridge] Fix coding style
[GelbooruBridge] Fix coding style
[GiphyBridge] Fix coding style
[GithubIssueBridge] Fix coding style
[GizmodoBridge] Fix coding style
[GoComicsBridge] Fix coding style
[GooglePlusPostBridge] Fix coding style
[GoogleSearchBridge] Fix coding style
[HDWallpapersBridge] Fix coding style
[HentaiHavenBridge] Fix coding style
[IdenticaBridge] Fix coding style
[InstagramBridge] Fix coding style
[IsoHuntBridge] Fix coding style
[JapanExpoBridge] Fix coding style
[KonachanBridge] Fix coding style
[KoreusBridge] Fix coding style
[KununuBridge] Fix coding style
[LeBonCoinBridge] Fix coding style
[LegifranceJOBBridge] Fix coding style
[LeMondeInformatiqueBridge] Fix coding style
[LesJoiesDuCodeBridge] Fix coding style
[LichessBridge] Fix coding style
[LinkedInCompanyBridge] Fix coding style
[LolibooruBridge] Fix coding style
[LWNprevBridge] Fix coding style
[MangareaderBridge] Fix coding style
[MilbooruBridge] Fix coding style
[MixCloudBridge] Fix coding style
[MoebooruBridge] Fix coding style
[MondeDiploBridge] Fix coding style
[MsnMondeBridge] Fix coding style
[MspabooruBridge] Fix coding style
[NasaApodBridge] Fix coding style
[NeuviemeArtBridge] Fix coding style
[NextgovBridge] Fix coding style
[NextInpactBridge] Fix coding style
[NiceMatinBridge] Fix coding style
[NovelUpdatesBridge] Fix coding style
[OpenClassroomsBridge] Fix coding style
[ParuVenduImmoBridge] Fix coding style
[PickyWallpapersBridge] Fix coding style
[PinterestBridge] Fix coding style
[PlanetLibreBridge] Fix coding style
[ReadComicsBridge] Fix coding style
[Releases3DSBridge] Fix coding style
[ReporterreBridge] Fix coding style
[RTBFBridge] Fix coding style
[Rue89Bridge] Fix coding style
[Rule34Bridge] Fix coding style
[Rule34pahealBridge] Fix coding style
[SafebooruBridge] Fix coding style
[SakugabooruBridge] Fix coding style
[ScmbBridge] Fix coding style
[ScoopItBridge] Fix coding style
[SensCritiqueBridge] Fix coding style
[SexactuBridge] Fix coding style
[ShanaprojectBridge] Fix coding style
[Shimmie2Bridge] Fix coding style
[SoundcloudBridge] Fix coding style
[StripeAPIChangeLogBridge] Fix coding style
[SuperbWallpapersBridge] Fix coding style
[T411Bridge] Fix coding style
[TagBoardBridge] Fix coding style
[TbibBridge] Fix coding style
[TheCodingLoveBridge] Fix coding style
[TheHackerNewsBridge] Fix coding style
[ThePirateBayBridge] Fix coding style
[TheTVDBBridge] Fix coding style
[Torrent9Bridge] Fix coding style
[TwitterBridge] Fix coding style
[UnsplashBridge] Fix coding style
[ViadeoCompanyBridge] Fix coding style
[VineBridge] Fix coding style
[VkBridge] Fix coding style
[WallpaperStopBridge] Fix coding style
[WebfailBridge] Fix coding style
[WeLiveSecurityBridge] Fix coding style
[WhydBridge] Fix coding style
[WikipediaBridge] Fix coding style
[WordPressBridge] Fix coding style
[WorldOfTanksBridge] Fix coding style
[XbooruBridge] Fix coding style
[YandereBridge] Fix coding style
[YoutubeBridge] Fix coding style
[ZDNetBridge] Fix coding style
2017-02-12 15:34:08 +01:00
logmanoriginal
22a7666d2b [phpcs] Include bridges for coding style checks 2017-02-12 15:34:08 +01:00
logmanoriginal
04b885264d Merge branch 'FixBridges' of https://github.com/logmanoriginal/rss-bridge 2017-02-11 12:40:31 +01:00
logmanoriginal
37b5df8985 Remove FlickrExploreBridge and FlickrTagBridge
These bridges are replaced by the FlickrBridge
2017-02-11 12:38:32 +01:00
logmanoriginal
f16835c223 [FlickrBridge] Add new bridge
This bridge is a mashup of the existing FlickrExploreBridge by sebsauvage
and FlickrTagBridge by erwang. It provides the same functionality as one
single bridge.
2017-02-11 12:36:08 +01:00
logmanoriginal
7ad8693b5f [FlickrTagBridge] Fix and improve bridge by using the FlickrExploreBridge approach 2017-02-11 12:20:44 +01:00
logmanoriginal
0f25684e65 [FlickrExplore] Fix and improve bridge
Instead of utilizing API requests for each element, the information
is now read directly from the source page, which provides information
as JSON data embedded in a script block.

The author name is returned for each element.

Improves the title and optionally adds the description if available
2017-02-11 12:03:41 +01:00
Teromene
9bf74b2715 Added the alternate facebook bridge. 2017-02-08 11:21:59 +00:00
logmanoriginal
d91c25cff1 Merge branch 'KernelBugTrackerBridge' of https://github.com/logmanoriginal/rss-bridge 2017-02-07 21:37:19 +01:00
logmanoriginal
6ddcedb53f [KernelBugTracker] Add new bridge
This adds a bridge for bugzilla.kernel.org to provide feeds for
bug comments without the need of registering an email address.

This implementation makes use of the print preview feature that
reduces bandwidth by a small margin.

Provides options to specify the number of comments to return as
well as the sorting order (latest first or oldest first)
2017-02-07 21:29:15 +01:00
Astyan-42
a1764a9fe2 make the bridge compatible with wordpress with no static URL (#469) 2017-02-07 10:24:18 +00:00
Pellaeon Lin
1028e538ab Fix duplicate https://www.facebook.com/ on captcha_action POST URL (#466) 2017-02-02 16:18:23 +00:00
Pellaeon Lin
49cc0661ad Fix FacebookBridge feed name empty when data loaded from cache (#456) 2017-02-02 16:17:18 +00:00
Pellaeon Lin
3109694b1c Ignore summary posts generated by facebook (#467) 2017-02-02 16:13:26 +00:00
Astyan-42
aa0a84bc26 adding theTVDBBridge (#463)
* adding theTVDBBridge
2017-02-01 13:02:05 +00:00
Teromene
eb22f86f44 Merge pull request #455 from pellaeon/pr-1
Preserve br and p from Facebook post content
2017-02-01 10:22:42 +00:00
Teromene
cf909ef3a1 Merge pull request #465 from JeremyRand/duckduckgo-fix-redirect
[DuckDuckGoBridge] Disable redirects
2017-02-01 10:15:57 +00:00
JeremyRand
94d2ebec0a [DuckDuckGoBridge] Disable DuckDuckGo redirects so that the links returned are correct. 2017-02-01 00:58:05 +00:00
Mitsu
44c7cbe2d7 Merge pull request #458 from ldidry/gocomicsbridge
Add GoComicsBridge
2017-01-14 11:04:01 +01:00
Luc Didry
5b4ba621ee Add GoComicsBridge 2017-01-14 10:01:00 +01:00
Pellaeon Lin
9c1bedb33f Preserve br and p from Facebook post content 2017-01-13 06:38:05 +08:00
LogMANOriginal
2fd60c68b0 Merge pull request #452 from lagaisse/lagaisse
Add Torrent9bridge
2017-01-12 21:12:28 +01:00
Badet Aurélien
670d8f18cb [BridgeAbstract] Enable caching of extraInfos - Issue #431 (#434)
Enable caching of extraInfos.
2017-01-03 10:28:47 +00:00
Kevin Lagaisse
41714b4c40 Add Torrent9bridge
update cpasbienbridge
2017-01-01 12:05:37 +01:00
LogMANOriginal
a4f4447c5e Merge pull request #450 from ORelio/master
Update Futura-Sciences field retrieval
2016-12-19 20:17:20 +01:00
logmanoriginal
3d984e8762 Merge branch 'FixBridges' of https://github.com/logmanoriginal/rss-bridge 2016-12-19 20:09:28 +01:00
ORelio
3a6ccc4c29 Update Futura-Sciences field retrieval
See #433
2016-12-19 20:04:34 +01:00
logmanoriginal
f45405950d [GooglePlusPost] Trim unnecessary tags 2016-12-19 19:57:25 +01:00
logmanoriginal
0e5cf0d14e [GooglePlusPost] Return content and title compatible to current RSS-Bridge 2016-12-19 19:49:53 +01:00
logmanoriginal
9405dc6c4b Merge branch 'FixBridges' of https://github.com/logmanoriginal/rss-bridge 2016-12-17 18:11:59 +01:00
logmanoriginal
d0c9397613 [Kununu] Fix source layout changed 2016-12-17 18:04:21 +01:00
logmanoriginal
5ad3198d71 [Kununu] Fix content check condition
Check if null instead of false according to:
http://simplehtmldom.sourceforge.net/manual.htm#section_find
2016-12-17 17:21:29 +01:00
logmanoriginal
83b5bbcc37 [Kununu] Use tabs for indentation 2016-12-17 17:11:58 +01:00
logmanoriginal
f694023f7d bridges: Return default values for getName and getURI 2016-12-17 17:03:09 +01:00
logmanoriginal
61b9c3eb48 Merge branch 'FixBridges' of https://github.com/logmanoriginal/rss-bridge 2016-12-17 16:48:47 +01:00
logmanoriginal
d4fb02b0d0 bridges: Set missing MAINTAINER based on blame command
Maintainer should be set for all bridges. Using git blame to
determine who provided the most code to the files. This is
obviously not a good solution, feel free to insert own names
2016-12-17 16:43:47 +01:00
logmanoriginal
95b99d42a4 bridges: Fix default return values and function scopes
getURI and getName should fall back to parent::getURI or
parent::getName respectively if it cannot build propper
return values.

Functions defined by bridges should be made private to
prevent confusion with inherited functions
2016-12-17 16:39:18 +01:00
LogMANOriginal
271c71d0ac Merge pull request #444 from niawag/patch-3
[ThePirateBay] Filter results by user status (VIP/Trusted/None)
2016-12-16 18:22:33 +01:00
LogMANOriginal
1e0cef8f7f Merge pull request #448 from chemel/amazon
[AmazonBridge] Add possibility to choose Amazon location
2016-12-16 18:19:19 +01:00
niawag
8b52b3858e Filter results by user status (VIP/Trusted/None)
Took into account LogMANOriginal comment and tested it, everything's working.
2016-12-16 10:41:40 +01:00
logmanoriginal
4a1e5245b3 [contents] Don't suppress errors returned by file_get_contents 2016-12-15 20:24:34 +01:00
LogMANOriginal
cad78be37b Merge pull request #445 from da2x/patch-1
Set “Accept-Encoding: gzip” header
2016-12-14 12:57:48 +01:00
Alexis CHEMEL
a5b0e2a24f [AmazonBridge] Adding country list box #447 2016-12-13 17:02:49 +01:00
Teromene
4972cec951 Merge pull request #446 from TwizzyDizzy/master
Translate amazon bridge to english
2016-12-12 21:34:35 +00:00
Thomas Dalichow
f09e8e1139 Translate amazon bridge to english
Until now, it was my understanding to keep rss-bridge english, not french. Despite it's french roots.
2016-12-12 22:28:05 +01:00
LogMANOriginal
64fa134c40 Merge pull request #439 from chemel/travis
Fix errors returned by travis in BridgeAbstract
2016-12-12 17:02:57 +01:00
LogMANOriginal
d9030bfb97 [readme]
Add travis build Status image
2016-12-12 17:02:05 +01:00
Daniel Aleksandersen
278d6a0ec2 Set “Accept-Encoding: gzip” header 2016-12-12 11:29:34 +01:00
niawag
8bb002c7b6 Filter results by user status (VIP/Trusted/None) 2016-12-11 12:11:03 +01:00
Teromene
b4e6c0d973 Merge pull request #443 from EtienneM/patch-2
Fix SCMB bridge
2016-12-10 15:26:33 +00:00
Étienne
1ef7e40ecd Fix SCMB bridge
This trailing `/` made the link to each article invalid
2016-12-10 15:56:30 +01:00
Teromene
00403214ce Merge pull request #441 from chemel/amazon-bridge
Amazon Bridge
2016-12-10 14:13:10 +00:00
Teromene
9c65c7b9e1 Merge pull request #442 from chemel/google
GoogleSearchBridge Fix
2016-12-10 14:12:37 +00:00
Alexis CHEMEL
877465d508 AmazonBridge 2016-12-10 14:58:48 +01:00
Alexis CHEMEL
35415004b9 Fix GoogleSearchBridge
find div instead li
2016-12-10 14:56:25 +01:00
Alexis CHEMEL
e908fe648b T411Bridge Minor fixes (#438)
T411Bridge Minor fixes
2016-12-06 17:20:41 +00:00
Alexis CHEMEL
3f503c4356 MixCloudBridge #436 (#437)
MixCloudBridge #436
FuturaSciencesBridge #433
2016-12-06 11:12:42 +00:00
Alexis CHEMEL
f4aa3b39e8 fix BridgeAbstract 2016-12-06 01:01:07 +01:00
Badet Aurélien
c702a0e69f Bridge getExtraInfos (#432)
* add function getExtraInfos() to BridgeAbstract

* replace call to $bridge->getName() and $bridge->getURI() by $bridge->getExtraInfos()

replace call to $bridge->getName() and $bridge->getURI() by $bridge->getExtraInfos() defined by default in BridgeAbstract.
So we could pass additionals ExtraInfos from custom bridges to custom formats.
2016-11-29 00:48:59 +00:00
logmanoriginal
5edba3a1aa Merge branch 'MultipleEnclosures' of https://github.com/logmanoriginal/rss-bridge 2016-11-16 16:55:14 +01:00
logmanoriginal
8d41718553 [HtmlFormat] Update stylesheet for attachments 2016-11-16 16:27:17 +01:00
logmanoriginal
72f40fbd75 [formats] Allow multiple enclosures
All formats now support multiple enclosures. RSS
will show a warning if more than one enclosure
is used since many feed reader don't support
multiple enclosures with RSS (also not clearly
specified in the specification)
2016-11-12 22:04:42 +01:00
logmanoriginal
14c689e7a3 [core] Fix typos 2016-11-09 19:10:40 +01:00
logmanoriginal
84bc9d2da6 Merge branch 'ImproveFormat' of https://github.com/logmanoriginal/rss-bridge 2016-11-09 19:01:17 +01:00
logmanoriginal
42cbc2e889 [formats] Fix enclosures
All bridges failed due to missing 'enclosures' element in
the items array. With this commit all formats (ATOM, RSS
and HTML) provide support for a single 'enclosure' element
2016-11-09 18:59:17 +01:00
logmanoriginal
3a2cb9ea1e [formats] Use custom characterset in all formats
The specified characterset will now apply to all formats
thus allowing other charactersets than 'UTF-8'
2016-11-09 18:43:06 +01:00
logmanoriginal
4f4fb11789 [FormatAbstract] Allow child classes to overwrite DEFAULT_CHARSET
By using 'static' instead of 'self' the constant may
be overridden by child classes.
2016-11-09 18:41:25 +01:00
logmanoriginal
28e813620f [FormatInterface] Add missing public functions
This commit adds all missing functions to the interface
that are defined and implemented as public functions in
FormatAbstract.
2016-11-09 18:41:24 +01:00
LogMANOriginal
fdf98041e3 Merge pull request #428 from mro/feature/xml_escape
CDATA considered harmful.
2016-11-09 18:35:52 +01:00
LogMANOriginal
29e64f77aa Merge pull request #430 from sysadminstory/master
Fix Cache write verification
2016-11-09 18:26:54 +01:00
Gilles Maurer
4dfbc16a5b Fix Cache write verification
PHP operator '===' is the only strict way to mix up the value '0' and
the value 'FALSE'.

The function saveData of the FileCache tests if the write of the cache
files was done with success and raise an Exception if not. The test was
done without the '===' operator, and if the data is 0 bytes long the
error message says there is a permission error, which is false.

A data 0 bytes long is another issue, either in the json_encode function
either in the Bridge, but not a permission issue.
2016-11-09 02:11:22 +01:00
Marcus Rohrmoser
af572341b3 Fix to PR #427 (#429)
* Ouch, sorry.

* fix indentation.
2016-11-08 21:44:54 +00:00
Teromene
51e9298a2b Merge pull request #427 from mro/feature/enclosure
Revive enclosures.
2016-11-08 21:24:44 +00:00
Marcus Rohrmoser
6df657179f CDATA considered harmful.
https://evertpot.com/184/
2016-11-08 22:21:48 +01:00
Marcus Rohrmoser
2ff422d312 Revive enclosures.Reverts 0663c95. Refs #198 #175. 2016-11-08 22:03:51 +01:00
Teromene
1b3efce64d Merge pull request #426 from mro/language
don’t lie about the language.
2016-11-08 00:33:17 +00:00
Marcus Rohrmoser
750812c512 don’t lie about the language. 2016-11-07 23:29:44 +01:00
201 changed files with 15438 additions and 6490 deletions

8
.dockerignore Normal file
View File

@@ -0,0 +1,8 @@
.git
cache/*
DEBUG
Dockerfile
whitelist.txt
phpcs.xml
CHANGELOG.md
CONTRIBUTING.md

6
.gitignore vendored
View File

@@ -227,8 +227,12 @@ pip-log.txt
/cache
/whitelist.txt
DEBUG
config.ini.php
######################
## VisualStudioCode ##
######################
.vscode/*
.vscode/*
#Builder
.buildconfig

View File

@@ -1,11 +1,9 @@
dist: trusty
sudo: false
language: php
php:
- '5.6'
- '7.0'
- hhvm
- nightly
install:
- pear channel-update pear.php.net
- pear install PHP_CodeSniffer
script:
@@ -14,6 +12,13 @@ script:
matrix:
fast_finish: true
include:
- php: 5.6
- php: 7.0
- php: hhvm
- php: nightly
allow_failures:
- php: hhvm
- php: nightly
- php: nightly

View File

@@ -1,11 +1,105 @@
rss-bridge Changelog
===
Alpha 0.1
===
* First tagged version.
* Includes refactoring.
* Unstable.
RSS-Bridge 2017-08-19
==
## General changes
* whitelist: Do case-insensitive whitelist matching
* [FeedExpander] Fix Serialization of 'SimpleXMLElement' is not allowed
* [FeedExpander] Remove whitespace from source content
* [index] Add GET parameter 'q' for search queries
- **Example**: You can now add `&q=Twitter` to load into the search field
* [index] Check permissions for cache folder and whitelist file
* [index] Show bridge options when loading with URL fragment
- **Example**: You can now add `#bridge-Twitter` to load the card with all
parameters visible
* [style] Center search cursor and hide placeholder
* [validation] Fix error on undefined optional numeric value
## Modified bridges
* [DanbooruBridge] Allow descendant classes to override tag collection
* [DribbbleBridge] Add dribble bridge listing last dribble popular shots (#558)
* [FacebookBridge] Fix &amp; in URLs
* [GelbooruBridge] Fix bridge not getting tags correctly
* [GoComicsBridge] Fix for page structure changes (#568)
* [LeBonCoinBridge] Fix bridge is marked executable
* [LWNprevBridge] Fix everchanging url
* [YoutubeBridge] Fix error on certain keywords
* [YoutubeBridge] Fix issues loading playlists
## Removed bridges
* VineBridge
RSS-Bridge 2017-08-03
==
## Important changes
* RSS-Bridge now has [contribution guidelines](CONTRIBUTING.md)
* [phpcs rules](phpcs.xml) follow the [contribution guidelines](CONTRIBUTING.md)
## General changes
* Added a search bar to make searching for bridges easier
* Added user friendly error page for when a bridge fails
* Added caching of extraInfos (name, uri)
* Added an indicator to warn for bridges using HTTP instead of HTTPS
* Various bug fixes and improvements
## Modified bridges
* AllocineFRBridge] Update Faux Raccord link
* [DanbooruBridge] Fix broken URI
* [DuckDuckGoBridge] Disable DuckDuckGo redirects so that the links returned are correct.
* [FacebookBridge] Add option to hide posts with facebook videos
* [FacebookBridge] Add requester languages to HTTP header
* [FacebookBridge] Handle summary posts
* [FacebookBridge] Replace 'novideo' with 'media_type'
* [FilterBridge] Initial implementation of basic title permit and block
* [FlickrTagBridge] Fix and improve bridge by using the FlickrExploreBridge approach
* [GooglePlusPostBridge] Autofix user names
* [GooglePlusPostBridge] Fix bridge implementation
* [GooglePlusPostBridge] Fix content loading
* [InstagramBridge] Add option to filter for videos and pictures
* [LWNprevBridge] full rewrite
* [MangareaderBridge] Fix double forward slashes
* [NasaApodBridge] Use HTTPS instead of HTTP
* [PinterestBridge] Fix checkbox not working
* [PinterestBridge] Fix implementation after DOM changes
* [RTBFBridge] Update URI
* [SexactuBridge] Fix URI and timestamp
* [SexactuBridge] Use most modern version of bridge api and cached pages (#504)
* [ShanaprojectBridge] Don't throw error if timestamp is missing
* [TwitterBridge] Add option to hide retweets
* [TwitterBridge] Avoid empty content caused by new login policy
* [TwitterBridge] Fix double slashes in URI
* [TwitterBridge] Fix missing spaces
* [TwitterBridge] Fix title includes anchors in plaintext format
* [TwitterBridge] ignore promoted tweets
* [TwitterBridge] Optimize returned image sizes
* [TwitterBridge] Show quotes and pictures
* [WebfailBridge] Properly handle gifs (DOM changed)
* [YoutubeBridge] Improve readability of feed contents
* [YoutubeBridge] Improve URL handling in video descriptions
## New bridges
* AmazonBridge
* DiceBridge
* EtsyBridge
* FB2Bridge
* FilterBridge
* FlickrBridge
* GithubSearchBridge
* GoComicsBridge
* KATBridge
* KernelBugTrackerBridge
* MixCloudBridge
* MoinMoinBridge
* RainbowSixSiegeBridge
* SteamBridge
* TheTVDBBridge
* Torrent9Bridge
* UsbekEtRicaBridge
* WikiLeaksBridge
* WordPressPluginUpdateBridge
Alpha 0.2
===
@@ -161,3 +255,9 @@ Alpha 0.2
* YandereBridge
* YoutubeBridge
* ZDNetBridge
Alpha 0.1
===
* First tagged version.
* Includes refactoring.
* Unstable.

47
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,47 @@
### Pull request policy
Fix one issue per pull request.
Squash commits before opening a pull request.
Respect the coding style policy.
Name your PR like the following :
* When correcting a single bridge, use `[BridgeName] Feature`.
* When fixing a problem in a specific file, use `[FileName] Feature`.
* When fixing a general problem, use `category : feature`.
Note that all pull-requests should pass the unit tests before they can be merged.
### Coding style
Use `camelCase` for variables and methods.
Use `UPPERCASE` for constants.
Use `PascalCase` for class names. When creating a bridge, your class and PHP file should be named `MyImplementationBridge`.
Use tabs for indentation.
Add an empty line at the end of your file.
Use `''` to encapsulate strings, including in arrays.
Prefer lines shorter than 80 chars, no line longer than 120 chars.
PHP constants should be in lower case (`true, false, null`...)
* Add spaces between the logical operator and your expressions (not needed for the `!` operator).
* Use `||` and `&&` instead of `or` and `and`.
* Add space between your condition and the opening bracket/closing bracket.
* Don't put a space between `if` and your bracket.
* Use `elseif` instead of `else if`.
* Add new lines in your conditions if they are containing more than one line.
* Example :
```PHP
if($a == true && $b) {
print($a);
} else if(!$b) {
$a = !$a;
$b = $b >> $a;
print($b);
} else {
print($b);
}
```

5
Dockerfile Normal file
View File

@@ -0,0 +1,5 @@
FROM ulsmith/alpine-apache-php7
COPY ./ /app/public/
RUN chown -R apache:root /app/public

View File

@@ -1,29 +1,29 @@
rss-bridge
===
[![LICENSE](https://img.shields.io/badge/license-UNLICENSE-blue.svg)](UNLICENSE)
[![LICENSE](https://img.shields.io/badge/license-UNLICENSE-blue.svg)](UNLICENSE) [![Build Status](https://travis-ci.org/RSS-Bridge/rss-bridge.svg?branch=master)](https://travis-ci.org/RSS-Bridge/rss-bridge)
rss-bridge is a PHP project capable of generating ATOM feeds for websites which don't have one.
Supported sites/pages (main)
===
* `FlickrExplore` : [Latest interesting images](http://www.flickr.com/explore) from Flickr
* `GoogleSearch` : Most recent results from Google Search
* `GooglePlus` : Most recent posts of user timeline
* `Twitter` : Return keyword/hashtag search or user timeline
* `Identi.ca` : Identica user timeline (Should be compatible with other Pump.io instances)
* `YouTube` : YouTube user channel, playlist or search
* `Cryptome` : Returns the most recent documents from [Cryptome.org](http://cryptome.org/)
* `DansTonChat`: Most recent quotes from [danstonchat.com](http://danstonchat.com/)
* `DuckDuckGo`: Most recent results from [DuckDuckGo.com](https://duckduckgo.com/)
* `Instagram`: Most recent photos from an Instagram user
* `OpenClassrooms`: Lastest tutorials from [fr.openclassrooms.com](http://fr.openclassrooms.com/)
* `Pinterest`: Most recent photos from user or search
* `ScmbBridge`: Newest stories from [secouchermoinsbete.fr](http://secouchermoinsbete.fr/)
* `Wikipedia`: highlighted articles from [Wikipedia](https://wikipedia.org/) in English, German, French or Esperanto
* `Bandcamp` : Returns last release from [bandcamp](https://bandcamp.com/) for a tag
* `ThePirateBay` : Returns the newest indexed torrents from [The Pirate Bay](https://thepiratebay.se/) with keywords
* `Facebook` : Returns the latest posts on a page or profile on [Facebook](https://facebook.com/)
* `Bandcamp` : Returns last release from [bandcamp](https://bandcamp.com/) for a tag
* `Cryptome` : Returns the most recent documents from [Cryptome.org](http://cryptome.org/)
* `DansTonChat`: Most recent quotes from [danstonchat.com](http://danstonchat.com/)
* `DuckDuckGo`: Most recent results from [DuckDuckGo.com](https://duckduckgo.com/)
* `Facebook` : Returns the latest posts on a page or profile on [Facebook](https://facebook.com/)
* `FlickrExplore` : [Latest interesting images](http://www.flickr.com/explore) from Flickr
* `GooglePlus` : Most recent posts of user timeline
* `GoogleSearch` : Most recent results from Google Search
* `Identi.ca` : Identica user timeline (Should be compatible with other Pump.io instances)
* `Instagram`: Most recent photos from an Instagram user
* `OpenClassrooms`: Lastest tutorials from [fr.openclassrooms.com](http://fr.openclassrooms.com/)
* `Pinterest`: Most recent photos from user or search
* `ScmbBridge`: Newest stories from [secouchermoinsbete.fr](http://secouchermoinsbete.fr/)
* `ThePirateBay` : Returns the newest indexed torrents from [The Pirate Bay](https://thepiratebay.se/) with keywords
* `Twitter` : Return keyword/hashtag search or user timeline
* `Wikipedia`: highlighted articles from [Wikipedia](https://wikipedia.org/) in English, German, French or Esperanto
* `YouTube` : YouTube user channel, playlist or search
Plus [many other bridges](bridges/) to enable, thanks to the community
@@ -31,11 +31,11 @@ Output format
===
Output format can take several forms:
* `Atom` : ATOM Feed, for use in RSS/Feed readers
* `Mrss` : MRSS Feed, for use in RSS/Feed readers
* `Json` : Json, for consumption by other applications.
* `Html` : Simple html page.
* `Plaintext` : raw text (php object, as returned by print_r)
* `Atom` : ATOM Feed, for use in RSS/Feed readers
* `Html` : Simple html page.
* `Json` : Json, for consumption by other applications.
* `Mrss` : MRSS Feed, for use in RSS/Feed readers
* `Plaintext` : raw text (php object, as returned by print_r)
Screenshot
===
@@ -53,7 +53,7 @@ Requirements
* PHP 5.6, e.g. `AddHandler application/x-httpd-php56 .php` in `.htaccess`
* `openssl` extension enabled in PHP config (`php.ini`)
* `allow_url_fopen=1` in `php.ini`
* `curl` extension enabled in PHP config (`php.ini`)
Enabling/Disabling bridges
===

View File

@@ -1,25 +1,42 @@
<?php
class ABCTabsBridge extends BridgeAbstract{
class ABCTabsBridge extends BridgeAbstract {
const MAINTAINER = "kranack";
const NAME = "ABC Tabs Bridge";
const URI = "http://www.abc-tabs.com/";
const DESCRIPTION = "Returns 22 newest tabs";
const MAINTAINER = 'kranack';
const NAME = 'ABC Tabs Bridge';
const URI = 'https://www.abc-tabs.com/';
const DESCRIPTION = 'Returns 22 newest tabs';
public function collectData(){
$html = '';
$html = getSimpleHTMLDOM(static::URI.'tablatures/nouveautes.html') or returnClientError('No results for this query.');
$html = getSimpleHTMLDOM(static::URI.'tablatures/nouveautes.html')
or returnClientError('No results for this query.');
$table = $html->find('table#myTable', 0)->children(1);
foreach ($table->find('tr') as $tab)
{
$item = array();
$item['author'] = $tab->find('td', 1)->plaintext . ' - ' . $tab->find('td', 2)->plaintext;
$item['title'] = $tab->find('td', 1)->plaintext . ' - ' . $tab->find('td', 2)->plaintext;
$item['content'] = 'Le ' . $tab->find('td', 0)->plaintext . '<br> Par: ' . $tab->find('td', 5)->plaintext . '<br> Type: ' . $tab->find('td', 3)->plaintext;
$item['id'] = static::URI . $tab->find('td', 2)->find('a', 0)->getAttribute('href');
$item['uri'] = static::URI . $tab->find('td', 2)->find('a', 0)->getAttribute('href');
$this->items[] = $item;
foreach ($table->find('tr') as $tab) {
$item = array();
$item['author'] = $tab->find('td', 1)->plaintext
. ' - '
. $tab->find('td', 2)->plaintext;
$item['title'] = $tab->find('td', 1)->plaintext
. ' - '
. $tab->find('td', 2)->plaintext;
$item['content'] = 'Le '
. $tab->find('td', 0)->plaintext
. '<br> Par: '
. $tab->find('td', 5)->plaintext
. '<br> Type: '
. $tab->find('td', 3)->plaintext;
$item['id'] = static::URI
. $tab->find('td', 2)->find('a', 0)->getAttribute('href');
$item['uri'] = static::URI
. $tab->find('td', 2)->find('a', 0)->getAttribute('href');
$this->items[] = $item;
}
}
}
}

View File

@@ -1,25 +1,25 @@
<?php
class AcrimedBridge extends FeedExpander {
const MAINTAINER = "qwertygc";
const NAME = "Acrimed Bridge";
const URI = "http://www.acrimed.org/";
const CACHE_TIMEOUT = 4800; //2hours
const DESCRIPTION = "Returns the newest articles.";
const MAINTAINER = 'qwertygc';
const NAME = 'Acrimed Bridge';
const URI = 'http://www.acrimed.org/';
const CACHE_TIMEOUT = 4800; //2hours
const DESCRIPTION = 'Returns the newest articles';
public function collectData(){
$this->collectExpandableDatas(static::URI.'spip.php?page=backend');
}
public function collectData(){
$this->collectExpandableDatas(static::URI . 'spip.php?page=backend');
}
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
$articlePage = getSimpleHTMLDOM($newsItem->link);
$article = sanitize($articlePage->find('article.article1', 0)->innertext);
$article = defaultImageSrcTo($article, static::URI);
$item['content'] = $article;
$articlePage = getSimpleHTMLDOM($newsItem->link);
$article = sanitize($articlePage->find('article.article1', 0)->innertext);
$article = defaultLinkTo($article, static::URI);
$item['content'] = $article;
return $item;
}
return $item;
}
}

View File

@@ -1,82 +1,87 @@
<?php
class AllocineFRBridge extends BridgeAbstract{
class AllocineFRBridge extends BridgeAbstract {
const MAINTAINER = 'superbaillot.net';
const NAME = 'Allo Cine Bridge';
const CACHE_TIMEOUT = 25200; // 7h
const URI = 'http://www.allocine.fr/';
const DESCRIPTION = 'Bridge for allocine.fr';
const PARAMETERS = array( array(
'category' => array(
'name' => 'category',
'type' => 'list',
'required' => true,
'exampleValue' => 'Faux Raccord',
'title' => 'Select your category',
'values' => array(
'Faux Raccord' => 'faux-raccord',
'Top 5' => 'top-5',
'Tueurs en Séries' => 'tueurs-en-serie'
)
)
));
const MAINTAINER = "superbaillot.net";
const NAME = "Allo Cine Bridge";
const CACHE_TIMEOUT = 25200; // 7h
const URI = "http://www.allocine.fr/";
const DESCRIPTION = "Bridge for allocine.fr";
const PARAMETERS = array( array(
'category'=>array(
'name'=>'category',
'type'=>'list',
'required'=>true,
'exampleValue'=>'Faux Raccord',
'title'=>'Select your category',
'values'=>array(
'Faux Raccord'=>'faux-raccord',
'Top 5'=>'top-5',
'Tueurs en Séries'=>'tueurs-en-serie'
)
)
));
public function getURI(){
if(!is_null($this->getInput('category'))) {
public function getURI(){
switch($this->getInput('category')){
case 'faux-raccord':
$uri = static::URI.'video/programme-12284/saison-27129/';
break;
case 'top-5':
$uri = static::URI.'video/programme-12299/saison-29561/';
break;
case 'tueurs-en-serie':
$uri = static::URI.'video/programme-12286/saison-22938/';
break;
}
switch($this->getInput('category')) {
case 'faux-raccord':
$uri = static::URI . 'video/programme-12284/saison-32180/';
break;
case 'top-5':
$uri = static::URI . 'video/programme-12299/saison-29561/';
break;
case 'tueurs-en-serie':
$uri = static::URI . 'video/programme-12286/saison-22938/';
break;
}
return $uri;
}
return $uri;
}
public function getName(){
return self::NAME.' : '
.array_search(
$this->getInput('category'),
self::PARAMETERS[$this->queriedContext]['category']['values']
);
}
return parent::getURI();
}
public function collectData(){
public function getName(){
if(!is_null($this->getInput('category'))) {
return self::NAME . ' : '
.array_search(
$this->getInput('category'),
self::PARAMETERS[$this->queriedContext]['category']['values']
);
}
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError("Could not request ".$this->getURI()." !");
return parent::getName();
}
$category=array_search(
$this->getInput('category'),
self::PARAMETERS[$this->queriedContext]['category']['values']
);
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request ' . $this->getURI() . ' !');
foreach($html->find('figure.media-meta-fig') as $element)
{
$item = array();
$category = array_search(
$this->getInput('category'),
self::PARAMETERS[$this->queriedContext]['category']['values']
);
$title = $element->find('div.titlebar h3.title a', 0);
$content = trim($element->innertext);
$figCaption = strpos($content, $category);
foreach($html->find('.media-meta-list figure.media-meta-fig') as $element) {
$item = array();
if($figCaption !== false)
{
$content = str_replace('src="/', 'src="'.static::URI, $content);
$content = str_replace('href="/', 'href="'.static::URI, $content);
$content = str_replace('src=\'/', 'src=\''.static::URI, $content);
$content = str_replace('href=\'/', 'href=\''.static::URI, $content);
$item['content'] = $content;
$item['title'] = trim($title->innertext);
$item['uri'] = static::URI . $title->href;
$this->items[] = $item;
}
}
}
$title = $element->find('div.titlebar h3.title a', 0);
$content = trim($element->innertext);
$figCaption = strpos($content, $category);
if($figCaption !== false) {
$content = str_replace('src="/', 'src="' . static::URI, $content);
$content = str_replace('href="/', 'href="' . static::URI, $content);
$content = str_replace('src=\'/', 'src=\'' . static::URI, $content);
$content = str_replace('href=\'/', 'href=\'' . static::URI, $content);
$item['content'] = $content;
$item['title'] = trim($title->innertext);
$item['uri'] = static::URI . $title->href;
$this->items[] = $item;
}
}
}
}

94
bridges/AmazonBridge.php Normal file
View File

@@ -0,0 +1,94 @@
<?php
class AmazonBridge extends BridgeAbstract {
const MAINTAINER = 'Alexis CHEMEL';
const NAME = 'Amazon';
const URI = 'https://www.amazon.com/';
const CACHE_TIMEOUT = 3600; // 1h
const DESCRIPTION = 'Returns products from Amazon search';
const PARAMETERS = array(array(
'q' => array(
'name' => 'Keyword',
'required' => true,
),
'sort' => array(
'name' => 'Sort by',
'type' => 'list',
'required' => false,
'values' => array(
'Relevance' => 'relevanceblender',
'Price: Low to High' => 'price-asc-rank',
'Price: High to Low' => 'price-desc-rank',
'Average Customer Review' => 'review-rank',
'Newest Arrivals' => 'date-desc-rank',
),
'defaultValue' => 'relevanceblender',
),
'tld' => array(
'name' => 'Country',
'type' => 'list',
'required' => true,
'values' => array(
'Australia' => 'com.au',
'Brazil' => 'com.br',
'Canada' => 'ca',
'China' => 'cn',
'France' => 'fr',
'Germany' => 'de',
'India' => 'in',
'Italy' => 'it',
'Japan' => 'co.jp',
'Mexico' => 'com.mx',
'Netherlands' => 'nl',
'Spain' => 'es',
'United Kingdom' => 'co.uk',
'United States' => 'com',
),
'defaultValue' => 'com',
),
));
public function getName(){
if(!is_null($this->getInput('tld')) && !is_null($this->getInput('q'))) {
return 'Amazon.'.$this->getInput('tld').': '.$this->getInput('q');
}
return parent::getName();
}
public function collectData() {
$uri = 'https://www.amazon.'.$this->getInput('tld').'/';
$uri .= 's/?field-keywords='.urlencode($this->getInput('q')).'&sort='.$this->getInput('sort');
$html = getSimpleHTMLDOM($uri)
or returnServerError('Could not request Amazon.');
foreach($html->find('li.s-result-item') as $element) {
$item = array();
// Title
$title = $element->find('h2', 0);
$item['title'] = html_entity_decode($title->innertext, ENT_QUOTES);
// Url
$uri = $title->parent()->getAttribute('href');
$uri = substr($uri, 0, strrpos($uri, '/'));
$item['uri'] = substr($uri, 0, strrpos($uri, '/'));
// Content
$image = $element->find('img', 0);
$price = $element->find('span.s-price', 0);
$price = ($price) ? $price->innertext : '';
$item['content'] = '<img src="'.$image->getAttribute('src').'" /><br />'.$price;
$this->items[] = $item;
}
}
}

View File

@@ -0,0 +1,149 @@
<?php
class AmazonPriceTrackerBridge extends BridgeAbstract {
const MAINTAINER = 'captn3m0';
const NAME = 'Amazon Price Tracker';
const URI = 'https://www.amazon.com/';
const CACHE_TIMEOUT = 3600; // 1h
const DESCRIPTION = 'Tracks price for a single product on Amazon';
const PARAMETERS = array(
array(
'asin' => array(
'name' => 'ASIN',
'required' => true,
'exampleValue' => 'B071GB1VMQ',
// https://stackoverflow.com/a/12827734
'pattern' => 'B[\dA-Z]{9}|\d{9}(X|\d)',
),
'tld' => array(
'name' => 'Country',
'type' => 'list',
'required' => true,
'values' => array(
'Australia' => 'com.au',
'Brazil' => 'com.br',
'Canada' => 'ca',
'China' => 'cn',
'France' => 'fr',
'Germany' => 'de',
'India' => 'in',
'Italy' => 'it',
'Japan' => 'co.jp',
'Mexico' => 'com.mx',
'Netherlands' => 'nl',
'Spain' => 'es',
'United Kingdom' => 'co.uk',
'United States' => 'com',
),
'defaultValue' => 'com',
),
));
protected $title;
/**
* Generates domain name given a amazon TLD
*/
private function getDomainName() {
return 'https://www.amazon.' . $this->getInput('tld');
}
/**
* Generates URI for a Amazon product page
*/
public function getURI() {
if (!is_null($this->getInput('asin'))) {
return $this->getDomainName() . '/dp/' . $this->getInput('asin') . '/';
}
return parent::getURI();
}
/**
* Scrapes the product title from the html page
* returns the default title if scraping fails
*/
private function getTitle($html) {
$titleTag = $html->find('#productTitle', 0);
if (!$titleTag) {
return $this->getDefaultTitle();
} else {
return trim(html_entity_decode($titleTag->innertext, ENT_QUOTES));
}
}
/**
* Title used by the feed if none could be found
*/
private function getDefaultTitle() {
return 'Amazon.' . $this->getInput('tld') . ': ' . $this->getInput('asin');
}
/**
* Returns name for the feed
* Uses title (already scraped) if it has one
*/
public function getName() {
if (isset($this->title)) {
return $this->title;
} else {
return parent::getName();
}
}
/**
* Returns a generated image tag for the product
*/
private function getImage($html) {
$imageSrc = $html->find('#main-image-container img', 0);
if ($imageSrc) {
$imageSrc = $imageSrc ? $imageSrc->getAttribute('data-old-hires') : '';
return <<<EOT
<img width="300" style="max-width:300;max-height:300" src="$imageSrc" alt="{$this->title}" />
EOT;
}
}
/**
* Return \simple_html_dom object
* for the entire html of the product page
*/
private function getHtml() {
$uri = $this->getURI();
return getSimpleHTMLDOM($uri) ?: returnServerError('Could not request Amazon.');
}
/**
* Scrape method for Amazon product page
* @return [type] [description]
*/
public function collectData() {
$html = $this->getHtml();
$this->title = $this->getTitle($html);
$imageTag = $this->getImage($html);
$asinData = $html->find('#cerberus-data-metrics', 0);
// <div id="cerberus-data-metrics" style="display: none;"
// data-asin="B00WTHJ5SU" data-asin-price="14.99" data-asin-shipping="0"
// data-asin-currency-code="USD" data-substitute-count="-1" ... />
$currency = $asinData->getAttribute('data-asin-currency-code');
$shipping = $asinData->getAttribute('data-asin-shipping');
$price = $asinData->getAttribute('data-asin-price');
$item = array(
'title' => $this->title,
'uri' => $this->getURI(),
'content' => "$imageTag<br/>Price: $price $currency",
);
if ($shipping !== '0') {
$item['content'] .= "<br>Shipping: $shipping $currency</br>";
}
$this->items[] = $item;
}
}

View File

@@ -1,119 +1,135 @@
<?php
class AnimeUltimeBridge extends BridgeAbstract {
const MAINTAINER = 'ORelio';
const NAME = 'Anime-Ultime';
const URI = 'http://www.anime-ultime.net/';
const CACHE_TIMEOUT = 10800; // 3h
const DESCRIPTION = 'Returns the 10 newest releases posted on Anime-Ultime';
const PARAMETERS = array( array(
'type'=>array(
'name'=>'Type',
'type'=>'list',
'values'=>array(
'Everything'=>'',
'Anime'=>'A',
'Drama'=>'D',
'Tokusatsu'=>'T'
)
)
));
const MAINTAINER = 'ORelio';
const NAME = 'Anime-Ultime';
const URI = 'http://www.anime-ultime.net/';
const CACHE_TIMEOUT = 10800; // 3h
const DESCRIPTION = 'Returns the 10 newest releases posted on Anime-Ultime';
const PARAMETERS = array( array(
'type' => array(
'name' => 'Type',
'type' => 'list',
'values' => array(
'Everything' => '',
'Anime' => 'A',
'Drama' => 'D',
'Tokusatsu' => 'T'
)
)
));
private $filter = 'Releases';
private $filter = 'Releases';
public function collectData(){
public function collectData(){
//Add type filter if provided
$typeFilter = array_search(
$this->getInput('type'),
self::PARAMETERS[$this->queriedContext]['type']['values']
);
//Add type filter if provided
$typeFilter = array_search(
$this->getInput('type'),
self::PARAMETERS[$this->queriedContext]['type']['values']
);
//Build date and filters for making requests
$thismonth = date('mY').$typeFilter;
$lastmonth = date('mY', mktime(0, 0, 0, date('n') - 1, 1, date('Y'))).$typeFilter;
//Build date and filters for making requests
$thismonth = date('mY') . $typeFilter;
$lastmonth = date('mY', mktime(0, 0, 0, date('n') - 1, 1, date('Y'))) . $typeFilter;
//Process each HTML page until having 10 releases
$processedOK = 0;
foreach (array($thismonth, $lastmonth) as $requestFilter) {
//Process each HTML page until having 10 releases
$processedOK = 0;
foreach (array($thismonth, $lastmonth) as $requestFilter) {
//Retrive page contents
$url = self::URI.'history-0-1/'.$requestFilter;
$html = getSimpleHTMLDOM($url)
or returnServerError('Could not request Anime-Ultime: '.$url);
//Retrive page contents
$url = self::URI . 'history-0-1/' . $requestFilter;
$html = getSimpleHTMLDOM($url)
or returnServerError('Could not request Anime-Ultime: ' . $url);
//Relases are sorted by day : process each day individually
foreach ($html->find('div.history', 0)->find('h3') as $daySection) {
//Relases are sorted by day : process each day individually
foreach($html->find('div.history', 0)->find('h3') as $daySection) {
//Retrieve day and build date information
$dateString = $daySection->plaintext;
$day = intval(substr($dateString, strpos($dateString, ' ') + 1, 2));
$item_date = strtotime(str_pad($day, 2, '0', STR_PAD_LEFT).'-'
.substr($requestFilter, 0, 2).'-'
.substr($requestFilter, 2, 4));
$release = $daySection->next_sibling()->next_sibling()->first_child(); //<h3>day</h3><br /><table><tr> <-- useful data in table rows
//Retrieve day and build date information
$dateString = $daySection->plaintext;
$day = intval(substr($dateString, strpos($dateString, ' ') + 1, 2));
$item_date = strtotime(str_pad($day, 2, '0', STR_PAD_LEFT)
. '-'
. substr($requestFilter, 0, 2)
. '-'
. substr($requestFilter, 2, 4));
//Process each release of that day, ignoring first table row: contains table headers
while (!is_null($release = $release->next_sibling())) {
if (count($release->find('td')) > 0) {
//<h3>day</h3><br /><table><tr> <-- useful data in table rows
$release = $daySection->next_sibling()->next_sibling()->first_child();
//Retrieve metadata from table columns
$item_link_element = $release->find('td', 0)->find('a', 0);
$item_uri = self::URI.$item_link_element->href;
$item_name = html_entity_decode($item_link_element->plaintext);
$item_episode = html_entity_decode(str_pad($release->find('td', 1)->plaintext, 2, '0', STR_PAD_LEFT));
$item_fansub = $release->find('td', 2)->plaintext;
$item_type = $release->find('td', 4)->plaintext;
//Process each release of that day, ignoring first table row: contains table headers
while(!is_null($release = $release->next_sibling())) {
if(count($release->find('td')) > 0) {
if (!empty($item_uri)) {
//Retrieve metadata from table columns
$item_link_element = $release->find('td', 0)->find('a', 0);
$item_uri = self::URI . $item_link_element->href;
$item_name = html_entity_decode($item_link_element->plaintext);
$item_episode = html_entity_decode(
str_pad(
$release->find('td', 1)->plaintext,
2,
'0',
STR_PAD_LEFT
)
);
//Retrieve description from description page and convert relative image src info absolute image src
$html_item = getContents($item_uri)
or returnServerError('Could not request Anime-Ultime: '.$item_uri);
$item_description = substr(
$html_item,
strpos($html_item, 'class="principal_contain" align="center">')
+ 41
);
$item_description = substr($item_description,
0,
strpos($item_description, '<div id="table">')
);
$item_description = str_replace(
'src="images', 'src="'.self::URI.'images',
$item_description
);
$item_description = str_replace("\r", '', $item_description);
$item_description = str_replace("\n", '', $item_description);
$item_description = utf8_encode($item_description);
$item_fansub = $release->find('td', 2)->plaintext;
$item_type = $release->find('td', 4)->plaintext;
//Build and add final item
$item = array();
$item['uri'] = $item_uri;
$item['title'] = $item_name.' '.$item_type.' '.$item_episode;
$item['author'] = $item_fansub;
$item['timestamp'] = $item_date;
$item['content'] = $item_description;
$this->items[] = $item;
$processedOK++;
if(!empty($item_uri)) {
//Stop processing once limit is reached
if ($processedOK >= 10)
return;
}
}
}
}
}
}
// Retrieve description from description page and
// convert relative image src info absolute image src
$html_item = getContents($item_uri)
or returnServerError('Could not request Anime-Ultime: ' . $item_uri);
$item_description = substr(
$html_item,
strpos($html_item, 'class="principal_contain" align="center">') + 41
);
$item_description = substr($item_description,
0,
strpos($item_description, '<div id="table">')
);
$item_description = str_replace(
'src="images', 'src="' . self::URI . 'images',
$item_description
);
$item_description = str_replace("\r", '', $item_description);
$item_description = str_replace("\n", '', $item_description);
$item_description = utf8_encode($item_description);
public function getName() {
$typeFilter = array_search(
$this->getInput('type'),
self::PARAMETERS[$this->queriedContext]['type']['values']
);
//Build and add final item
$item = array();
$item['uri'] = $item_uri;
$item['title'] = $item_name . ' ' . $item_type . ' ' . $item_episode;
$item['author'] = $item_fansub;
$item['timestamp'] = $item_date;
$item['content'] = $item_description;
$this->items[] = $item;
$processedOK++;
return 'Latest '.$typeFilter.' - Anime-Ultime Bridge';
}
//Stop processing once limit is reached
if ($processedOK >= 10)
return;
}
}
}
}
}
}
public function getName() {
if(!is_null($this->getInput('type'))) {
$typeFilter = array_search(
$this->getInput('type'),
self::PARAMETERS[$this->queriedContext]['type']['values']
);
return 'Latest ' . $typeFilter . ' - Anime-Ultime Bridge';
}
return parent::getName();
}
}

View File

@@ -1,91 +1,100 @@
<?php
class Arte7Bridge extends BridgeAbstract{
class Arte7Bridge extends BridgeAbstract {
const MAINTAINER = "mitsukarenai";
const NAME = "Arte +7";
const URI = "http://www.arte.tv/";
const CACHE_TIMEOUT = 1800; // 30min
const DESCRIPTION = "Returns newest videos from ARTE +7";
const PARAMETERS = array(
'Catégorie (Français)' => array(
'catfr'=>array(
'type'=>'list',
'name'=>'Catégorie',
'values'=>array(
'Toutes les vidéos (français)'=>'toutes-les-videos',
'Actu & société'=>'actu-société',
'Séries & fiction'=>'séries-fiction',
'Cinéma'=>'cinéma',
'Arts & spectacles classiques'=>'arts-spectacles-classiques',
'Culture pop'=>'culture-pop',
'Découverte'=>'découverte',
'Histoire'=>'histoire',
'Junior'=>'junior'
const MAINTAINER = 'mitsukarenai';
const NAME = 'Arte +7';
const URI = 'https://www.arte.tv/';
const CACHE_TIMEOUT = 1800; // 30min
const DESCRIPTION = 'Returns newest videos from ARTE +7';
)
)
),
'Catégorie (Allemand)' => array(
'catde'=>array(
'type'=>'list',
'name'=>'Catégorie',
'values'=>array(
'Alle Videos (deutsch)'=>'alle-videos',
'Aktuelles & Gesellschaft'=>'aktuelles-gesellschaft',
'Fernsehfilme & Serien'=>'fernsehfilme-serien',
'Kino'=>'kino',
'Kunst & Kultur'=>'kunst-kultur',
'Popkultur & Alternativ'=>'popkultur-alternativ',
'Entdeckung'=>'entdeckung',
'Geschichte'=>'geschichte',
'Junior'=>'junior'
)
)
)
);
const API_TOKEN = 'Nzc1Yjc1ZjJkYjk1NWFhN2I2MWEwMmRlMzAzNjI5NmU3NWU3ODg4ODJjOWMxNTMxYzEzZGRjYjg2ZGE4MmIwOA';
public function collectData(){
const PARAMETERS = array(
'Catégorie (Français)' => array(
'catfr' => array(
'type' => 'list',
'name' => 'Catégorie',
'values' => array(
'Toutes les vidéos (français)' => null,
'Actu & société' => 'ACT',
'Séries & fiction' => 'SER',
'Cinéma' => 'CIN',
'Arts & spectacles classiques' => 'ARS',
'Culture pop' => 'CPO',
'Découverte' => 'DEC',
'Histoire' => 'HIST',
'Science' => 'SCI',
'Autre' => 'AUT'
)
)
),
'Catégorie (Allemand)' => array(
'catde' => array(
'type' => 'list',
'name' => 'Catégorie',
'values' => array(
'Alle Videos (deutsch)' => null,
'Aktuelles & Gesellschaft' => 'ACT',
'Fernsehfilme & Serien' => 'SER',
'Kino' => 'CIN',
'Kunst & Kultur' => 'ARS',
'Popkultur & Alternativ' => 'CPO',
'Entdeckung' => 'DEC',
'Geschichte' => 'HIST',
'Wissenschaft' => 'SCI',
'Sonstiges' => 'AUT'
)
)
)
);
switch($this->queriedContext){
case 'Catégorie (Français)':
$category=$this->getInput('catfr');
$lang='fr';
break;
case 'Catégorie (Allemand)':
$category=$this->getInput('catde');
$lang='de';
break;
}
public function collectData(){
switch($this->queriedContext) {
case 'Catégorie (Français)':
$category = $this->getInput('catfr');
$lang = 'fr';
break;
case 'Catégorie (Allemand)':
$category = $this->getInput('catde');
$lang = 'de';
break;
}
$url = self::URI.'guide/'.$lang.'/plus7/'.$category;
$input = getContents($url) or die('Could not request ARTE.');
if(strpos($input, 'categoryVideoSet') !== FALSE){
$input = explode('categoryVideoSet="', $input);
$input = explode('}}', $input[1]);
$input = $input[0].'}}';
}else{
$input = explode('videoSet="', $input);
$input = explode('}]}', $input[1]);
$input = $input[0].'}]}';
}
$url = 'https://api.arte.tv/api/opa/v3/videos?sort=-lastModified&limit=10&language='
. $lang
. ($category != null ? '&category.code=' . $category : '');
$input_json = json_decode(html_entity_decode($input, ENT_QUOTES), TRUE);
$header = array(
'Authorization: Bearer ' . self::API_TOKEN
);
foreach($input_json['videos'] as $element) {
$item = array();
$item['uri'] = str_replace("autoplay=1", "", $element['url']);
$item['id'] = $element['id'];
$hack_broadcast_time = $element['rights_end'];
$hack_broadcast_time = strtok($hack_broadcast_time, 'T');
$hack_broadcast_time = strtok('T');
$item['timestamp'] = strtotime($element['scheduled_on'].'T'.$hack_broadcast_time);
$item['title'] = $element['title'];
if (!empty($element['subtitle']))
$item['title'] = $element['title'].' | '.$element['subtitle'];
$item['duration'] = round((int)$element['duration']/60);
$item['content'] = $element['teaser'].'<br><br>'.$item['duration'].'min<br><a href="'.$item['uri'].'"><img src="' . $element['thumbnail_url'] . '" /></a>';
$this->items[] = $item;
}
}
$input = getContents($url, $header) or die('Could not request ARTE.');
$input_json = json_decode($input, true);
foreach($input_json['videos'] as $element) {
$item = array();
$item['uri'] = $element['url'];
$item['id'] = $element['id'];
$item['timestamp'] = strtotime($element['videoRightsBegin']);
$item['title'] = $element['title'];
if(!empty($element['subtitle']))
$item['title'] = $element['title'] . ' | ' . $element['subtitle'];
$item['duration'] = round((int)$element['durationSeconds'] / 60);
$item['content'] = $element['teaserText']
. '<br><br>'
. $item['duration']
. 'min<br><a href="'
. $item['uri']
. '"><img src="'
. $element['mainImage']['url']
. '" /></a>';
$this->items[] = $item;
}
}
}

View File

@@ -1,52 +1,74 @@
<?php
class AskfmBridge extends BridgeAbstract{
class AskfmBridge extends BridgeAbstract {
const MAINTAINER = "az5he6ch";
const NAME = "Ask.fm Answers";
const URI = "http://ask.fm/";
const CACHE_TIMEOUT = 300; //5 min
const DESCRIPTION = "Returns answers from an Ask.fm user";
const PARAMETERS = array(
'Ask.fm username'=>array(
'u'=>array(
'name'=>'Username',
'required'=>true
)
)
);
const MAINTAINER = 'az5he6ch';
const NAME = 'Ask.fm Answers';
const URI = 'https://ask.fm/';
const CACHE_TIMEOUT = 300; //5 min
const DESCRIPTION = 'Returns answers from an Ask.fm user';
const PARAMETERS = array(
'Ask.fm username' => array(
'u' => array(
'name' => 'Username',
'required' => true
)
)
);
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Requested username can\'t be found.');
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Requested username can\'t be found.');
foreach($html->find('div.streamItem-answer') as $element) {
$item = array();
$item['uri'] = self::URI.$element->find('a.streamItemsAge',0)->href;
$question = trim($element->find('h1.streamItemContent-question',0)->innertext);
$item['title'] = trim(htmlspecialchars_decode($element->find('h1.streamItemContent-question',0)->plaintext, ENT_QUOTES));
$answer = trim($element->find('p.streamItemContent-answer',0)->innertext);
#$item['update'] = $element->find('a.streamitemsage',0)->data-hint; // Doesn't work, DOM parser doesn't seem to like data-hint, dunno why
$visual = $element->find('div.streamItemContent-visual',0)->innertext; // This probably should be cleaned up, especially for YouTube embeds
//Fix tracking links, also doesn't work
foreach($element->find('a') as $link) {
if (strpos($link->href, 'l.ask.fm') !== false) {
#$link->href = str_replace('#_=_', '', get_headers($link->href, 1)['Location']); // Too slow
$link->href = $link->plaintext;
}
}
$content = '<p>' . $question . '</p><p>' . $answer . '</p><p>' . $visual . '</p>';
// Fix relative links without breaking // scheme used by YouTube stuff
$content = preg_replace('#href="\/(?!\/)#', 'href="'.self::URI,$content);
$item['content'] = $content;
$this->items[] = $item;
}
}
foreach($html->find('div.streamItem-answer') as $element) {
$item = array();
$item['uri'] = self::URI . $element->find('a.streamItemsAge', 0)->href;
$question = trim($element->find('h1.streamItemContent-question', 0)->innertext);
public function getName(){
return self::NAME.' : '.$this->getInput('u');
}
$item['title'] = trim(
htmlspecialchars_decode($element->find('h1.streamItemContent-question', 0)->plaintext,
ENT_QUOTES
)
);
public function getURI(){
return self::URI.urlencode($this->getInput('u')).'/answers/more?page=0';
}
$answer = trim($element->find('p.streamItemContent-answer', 0)->innertext);
// Doesn't work, DOM parser doesn't seem to like data-hint, dunno why
#$item['update'] = $element->find('a.streamitemsage',0)->data-hint;
// This probably should be cleaned up, especially for YouTube embeds
$visual = $element->find('div.streamItemContent-visual', 0)->innertext;
//Fix tracking links, also doesn't work
foreach($element->find('a') as $link) {
if(strpos($link->href, 'l.ask.fm') !== false) {
// Too slow
#$link->href = str_replace('#_=_', '', get_headers($link->href, 1)['Location']);
$link->href = $link->plaintext;
}
}
$content = '<p>' . $question . '</p><p>' . $answer . '</p><p>' . $visual . '</p>';
// Fix relative links without breaking // scheme used by YouTube stuff
$content = preg_replace('#href="\/(?!\/)#', 'href="' . self::URI, $content);
$item['content'] = $content;
$this->items[] = $item;
}
}
public function getName(){
if(!is_null($this->getInput('u'))) {
return self::NAME . ' : ' . $this->getInput('u');
}
return parent::getName();
}
public function getURI(){
if(!is_null($this->getInput('u'))) {
return self::URI . urlencode($this->getInput('u')) . '/answers/more?page=0';
}
return parent::getURI();
}
}

View File

@@ -1,43 +1,63 @@
<?php
class BandcampBridge extends BridgeAbstract{
class BandcampBridge extends BridgeAbstract {
const MAINTAINER = "sebsauvage";
const NAME = "Bandcamp Tag";
const URI = "http://bandcamp.com/";
const CACHE_TIMEOUT = 600; // 10min
const DESCRIPTION = "New bandcamp release by tag";
const PARAMETERS = array( array(
'tag'=>array(
'name'=>'tag',
'type'=>'text',
'required'=>true
)
));
const MAINTAINER = 'sebsauvage';
const NAME = 'Bandcamp Tag';
const URI = 'https://bandcamp.com/';
const CACHE_TIMEOUT = 600; // 10min
const DESCRIPTION = 'New bandcamp release by tag';
const PARAMETERS = array( array(
'tag' => array(
'name' => 'tag',
'type' => 'text',
'required' => true
)
));
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('No results for this query.');
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('No results for this query.');
foreach($html->find('li.item') as $release) {
$script = $release->find('div.art', 0)->getAttribute('onclick');
$uri = ltrim($script, "return 'url(");
$uri = rtrim($uri, "')");
foreach($html->find('li.item') as $release) {
$script = $release->find('div.art', 0)->getAttribute('onclick');
$uri = ltrim($script, "return 'url(");
$uri = rtrim($uri, "')");
$item = array();
$item['author'] = $release->find('div.itemsubtext',0)->plaintext . ' - ' . $release->find('div.itemtext',0)->plaintext;
$item['title'] = $release->find('div.itemsubtext',0)->plaintext . ' - ' . $release->find('div.itemtext',0)->plaintext;
$item['content'] = '<img src="' . $uri . '"/><br/>' . $release->find('div.itemsubtext',0)->plaintext . ' - ' . $release->find('div.itemtext',0)->plaintext;
$item['id'] = $release->find('a',0)->getAttribute('href');
$item['uri'] = $release->find('a',0)->getAttribute('href');
$this->items[] = $item;
}
}
$item = array();
$item['author'] = $release->find('div.itemsubtext', 0)->plaintext
. ' - '
. $release->find('div.itemtext', 0)->plaintext;
public function getURI(){
return self::URI.'tag/'.urlencode($this->getInput('tag')).'?sort_field=date';
}
$item['title'] = $release->find('div.itemsubtext', 0)->plaintext
. ' - '
. $release->find('div.itemtext', 0)->plaintext;
public function getName(){
return $this->getInput('tag') .' - '.'Bandcamp Tag';
}
$item['content'] = '<img src="'
. $uri
. '"/><br/>'
. $release->find('div.itemsubtext', 0)->plaintext
. ' - '
. $release->find('div.itemtext', 0)->plaintext;
$item['id'] = $release->find('a', 0)->getAttribute('href');
$item['uri'] = $release->find('a', 0)->getAttribute('href');
$this->items[] = $item;
}
}
public function getURI(){
if(!is_null($this->getInput('tag'))) {
return self::URI . 'tag/' . urlencode($this->getInput('tag')) . '?sort_field=date';
}
return parent::getURI();
}
public function getName(){
if(!is_null($this->getInput('tag'))) {
return $this->getInput('tag') . ' - Bandcamp Tag';
}
return parent::getName();
}
}

View File

@@ -1,19 +1,22 @@
<?php
class BastaBridge extends BridgeAbstract{
const MAINTAINER = "qwertygc";
const NAME = "Bastamag Bridge";
const URI = "http://www.bastamag.net/";
class BastaBridge extends BridgeAbstract {
const MAINTAINER = 'qwertygc';
const NAME = 'Bastamag Bridge';
const URI = 'http://www.bastamag.net/';
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = "Returns the newest articles.";
const DESCRIPTION = 'Returns the newest articles.';
public function collectData(){
// Replaces all relative image URLs by absolute URLs. Relative URLs always start with 'local/'!
function ReplaceImageUrl($content){
return preg_replace('/src=["\']{1}([^"\']+)/ims', 'src=\''.self::URI.'$1\'', $content);
// Replaces all relative image URLs by absolute URLs.
// Relative URLs always start with 'local/'!
function replaceImageUrl($content){
return preg_replace('/src=["\']{1}([^"\']+)/ims', 'src=\'' . self::URI . '$1\'', $content);
}
$html = getSimpleHTMLDOM(self::URI.'spip.php?page=backend')
or returnServerError('Could not request Bastamag.');
$html = getSimpleHTMLDOM(self::URI . 'spip.php?page=backend')
or returnServerError('Could not request Bastamag.');
$limit = 0;
foreach($html->find('item') as $element) {
@@ -22,11 +25,10 @@ class BastaBridge extends BridgeAbstract{
$item['title'] = $element->find('title', 0)->innertext;
$item['uri'] = $element->find('guid', 0)->plaintext;
$item['timestamp'] = strtotime($element->find('dc:date', 0)->plaintext);
$item['content'] = ReplaceImageUrl(getSimpleHTMLDOM($item['uri'])->find('div.texte', 0)->innertext);
$item['content'] = replaceImageUrl(getSimpleHTMLDOM($item['uri'])->find('div.texte', 0)->innertext);
$this->items[] = $item;
$limit++;
}
}
}
}
?>

View File

@@ -1,33 +1,31 @@
<?php
class BlaguesDeMerdeBridge extends BridgeAbstract{
class BlaguesDeMerdeBridge extends BridgeAbstract {
const MAINTAINER = "superbaillot.net";
const NAME = "Blagues De Merde";
const URI = "http://www.blaguesdemerde.fr/";
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = "Blagues De Merde";
const MAINTAINER = 'superbaillot.net';
const NAME = 'Blagues De Merde';
const URI = 'http://www.blaguesdemerde.fr/';
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = 'Blagues De Merde';
public function collectData(){
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not request BDM.');
public function collectData(){
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not request BDM.');
foreach($html->find('article.joke_contener') as $element) {
$item = array();
$temp = $element->find('a');
foreach($html->find('article.joke_contener') as $element) {
$item = array();
$temp = $element->find('a');
if(isset($temp[2]))
{
$item['content'] = trim($element->find('div.joke_text_contener', 0)->innertext);
$uri = $temp[2]->href;
$item['uri'] = $uri;
$item['title'] = substr($uri, (strrpos($uri, "/") + 1));
$date = $element->find("li.bdm_date",0)->innertext;
$time = mktime(0, 0, 0, substr($date, 3, 2), substr($date, 0, 2), substr($date, 6, 4));
$item['timestamp'] = $time;
$item['author'] = $element->find("li.bdm_pseudo",0)->innertext;;
$this->items[] = $item;
}
}
}
if(isset($temp[2])) {
$item['content'] = trim($element->find('div.joke_text_contener', 0)->innertext);
$uri = $temp[2]->href;
$item['uri'] = $uri;
$item['title'] = substr($uri, (strrpos($uri, '/') + 1));
$date = $element->find('li.bdm_date', 0)->innertext;
$time = mktime(0, 0, 0, substr($date, 3, 2), substr($date, 0, 2), substr($date, 6, 4));
$item['timestamp'] = $time;
$item['author'] = $element->find('li.bdm_pseudo', 0)->innertext;
$this->items[] = $item;
}
}
}
}
?>

View File

@@ -0,0 +1,65 @@
<?php
class BloombergBridge extends BridgeAbstract
{
const NAME = 'Bloomberg';
const URI = 'https://www.bloomberg.com/';
const DESCRIPTION = 'Trending stories from Bloomberg';
const MAINTAINER = 'mdemoss';
const PARAMETERS = array(
'Trending Stories' => array(),
'From Search' => array(
'q' => array(
'name' => 'Keyword',
'required' => true
)
)
);
public function getName()
{
switch($this->queriedContext) {
case 'Trending Stories':
return self::NAME . ' Trending Stories';
case 'From Search':
if (!is_null($this->getInput('q'))) {
return self::NAME . ' Search : ' . $this->getInput('q');
}
break;
}
return parent::getName();
}
public function collectData()
{
switch($this->queriedContext) {
case 'Trending Stories': // Get list of top new <article>s from the front page.
$html = getSimpleHTMLDOMCached($this->getURI(), 300);
$stories = $html->find('ul.top-news-v3__stories article.top-news-v3-story');
break;
case 'From Search': // Get list of <article> elements from search.
$html = getSimpleHTMLDOMCached(
$this->getURI() .
'search?sort=time:desc&page=1&query=' .
urlencode($this->getInput('q')), 300
);
$stories = $html->find('div.search-result-items article.search-result-story');
break;
}
foreach ($stories as $element) {
$item['uri'] = $element->find('h1 a', 0)->href;
if (preg_match('#^https://#i', $item['uri']) !== 1) {
$item['uri'] = $this->getURI() . $item['uri'];
}
$articleHtml = getSimpleHTMLDOMCached($item['uri']);
if (!$articleHtml) {
continue;
}
$item['title'] = $element->find('h1 a', 0)->plaintext;
$item['timestamp'] = strtotime($articleHtml->find('meta[name=iso-8601-publish-date],meta[name=date]', 0)->content);
$item['content'] = $articleHtml->find('meta[name=description]', 0)->content;
$this->items[] = $item;
}
}
}

View File

@@ -1,36 +1,45 @@
<?php
require_once('GelbooruBridge.php');
class BooruprojectBridge extends GelbooruBridge{
class BooruprojectBridge extends GelbooruBridge {
const MAINTAINER = "mitsukarenai";
const NAME = "Booruproject";
const URI = "http://booru.org/";
const DESCRIPTION = "Returns images from given page of booruproject";
const MAINTAINER = 'mitsukarenai';
const NAME = 'Booruproject';
const URI = 'http://booru.org/';
const DESCRIPTION = 'Returns images from given page of booruproject';
const PARAMETERS = array(
'global' => array(
'p' => array(
'name' => 'page',
'type' => 'number'
),
't' => array(
'name' => 'tags'
)
),
'Booru subdomain (subdomain.booru.org)' => array(
'i' => array(
'name' => 'Subdomain',
'required' => true
)
)
);
const PARAMETERS = array(
'global'=>array(
'p'=>array(
'name'=>'page',
'type'=>'number'
),
't'=>array('name'=>'tags')
),
'Booru subdomain (subdomain.booru.org)'=>array(
'i'=>array(
'name'=>'Subdomain',
'required'=>true
)
)
);
const PIDBYPAGE = 20;
const PIDBYPAGE=20;
public function getURI(){
if(!is_null($this->getInput('i'))) {
return 'http://' . $this->getInput('i') . '.booru.org/';
}
public function getURI(){
return 'http://'.$this->getInput('i').'.booru.org/';
}
return parent::getURI();
}
public function getName(){
return static::NAME . ' ' . $this->getInput('i');
}
public function getName(){
if(!is_null($this->getInput('i'))) {
return static::NAME . ' ' . $this->getInput('i');
}
return parent::getName();
}
}

View File

@@ -1,10 +1,10 @@
<?php
class CADBridge extends FeedExpander {
const MAINTAINER = "nyutag";
const NAME = "CAD Bridge";
const URI = "http://www.cad-comic.com/";
const MAINTAINER = 'nyutag';
const NAME = 'CAD Bridge';
const URI = 'http://www.cad-comic.com/';
const CACHE_TIMEOUT = 7200; //2h
const DESCRIPTION = "Returns the newest articles.";
const DESCRIPTION = 'Returns the newest articles.';
public function collectData(){
$this->collectExpandableDatas('http://cdn2.cad-comic.com/rss.xml', 10);
@@ -12,35 +12,34 @@ class CADBridge extends FeedExpander {
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
$item['content'] = $this->CADExtractContent($item['uri']);
$item['content'] = $this->extractCADContent($item['uri']);
return $item;
}
private function CADExtractContent($url) {
private function extractCADContent($url) {
$html3 = getSimpleHTMLDOMCached($url);
// The request might fail due to missing https support or wrong URL
if($html3 == false)
return 'Daily comic not released yet';
$htmlpart = explode("/", $url);
$htmlpart = explode('/', $url);
switch ($htmlpart[3]){
switch ($htmlpart[3]) {
case 'cad':
preg_match_all("/http:\/\/cdn2\.cad-comic\.com\/comics\/cad-\S*png/", $html3, $url2);
preg_match_all('/http:\/\/cdn2\.cad-comic\.com\/comics\/cad-\S*png/', $html3, $url2);
break;
case 'sillies':
preg_match_all("/http:\/\/cdn2\.cad-comic\.com\/comics\/sillies-\S*gif/", $html3, $url2);
preg_match_all('/http:\/\/cdn2\.cad-comic\.com\/comics\/sillies-\S*gif/', $html3, $url2);
break;
default:
return 'Daily comic not released yet';
}
$img = implode ($url2[0]);
$img = implode($url2[0]);
$html3->clear();
unset ($html3);
unset($html3);
if ($img == '')
return 'Daily comic not released yet';
return '<img src="'.$img.'"/>';
return '<img src="' . $img . '"/>';
}
}
?>

View File

@@ -1,76 +1,93 @@
<?php
class CNETBridge extends BridgeAbstract {
const MAINTAINER = 'ORelio';
const NAME = 'CNET News';
const URI = 'http://www.cnet.com/';
const CACHE_TIMEOUT = 1800; // 30min
const DESCRIPTION = 'Returns the newest articles. <br /> You may specify a topic found in some section URLs, else all topics are selected.';
const MAINTAINER = 'ORelio';
const NAME = 'CNET News';
const URI = 'http://www.cnet.com/';
const CACHE_TIMEOUT = 1800; // 30min
const DESCRIPTION = 'Returns the newest articles. <br /> You may specify a
topic found in some section URLs, else all topics are selected.';
const PARAMETERS = array( array(
'topic'=>array('name'=>'Topic name')
));
const PARAMETERS = array( array(
'topic' => array(
'name' => 'Topic name'
)
));
public function collectData(){
public function collectData(){
function ExtractFromDelimiters($string, $start, $end) {
if (strpos($string, $start) !== false) {
$section_retrieved = substr($string, strpos($string, $start) + strlen($start));
$section_retrieved = substr($section_retrieved, 0, strpos($section_retrieved, $end));
return $section_retrieved;
} return false;
}
function extractFromDelimiters($string, $start, $end){
if(strpos($string, $start) !== false) {
$section_retrieved = substr($string, strpos($string, $start) + strlen($start));
$section_retrieved = substr($section_retrieved, 0, strpos($section_retrieved, $end));
return $section_retrieved;
}
function StripWithDelimiters($string, $start, $end) {
while (strpos($string, $start) !== false) {
$section_to_remove = substr($string, strpos($string, $start));
$section_to_remove = substr($section_to_remove, 0, strpos($section_to_remove, $end) + strlen($end));
$string = str_replace($section_to_remove, '', $string);
} return $string;
}
return false;
}
function CleanArticle($article_html) {
$article_html = '<p>'.substr($article_html, strpos($article_html, '<p>') + 3);
$article_html = StripWithDelimiters($article_html, '<span class="credit">', '</span>');
$article_html = StripWithDelimiters($article_html, '<script', '</script>');
$article_html = StripWithDelimiters($article_html, '<div class="shortcode related-links', '</div>');
$article_html = StripWithDelimiters($article_html, '<a class="clickToEnlarge">', '</a>');
return $article_html;
}
function stripWithDelimiters($string, $start, $end){
while(strpos($string, $start) !== false) {
$section_to_remove = substr($string, strpos($string, $start));
$section_to_remove = substr($section_to_remove, 0, strpos($section_to_remove, $end) + strlen($end));
$string = str_replace($section_to_remove, '', $string);
}
$pageUrl = self::URI.(empty($this->getInput('topic')) ? '' : 'topics/'.$this->getInput('topic').'/');
$html = getSimpleHTMLDOM($pageUrl) or returnServerError('Could not request CNET: '.$pageUrl);
$limit = 0;
return $string;
}
foreach($html->find('div.assetBody') as $element) {
if ($limit < 8) {
function cleanArticle($article_html){
$article_html = '<p>' . substr($article_html, strpos($article_html, '<p>') + 3);
$article_html = stripWithDelimiters($article_html, '<span class="credit">', '</span>');
$article_html = stripWithDelimiters($article_html, '<script', '</script>');
$article_html = stripWithDelimiters($article_html, '<div class="shortcode related-links', '</div>');
$article_html = stripWithDelimiters($article_html, '<a class="clickToEnlarge">', '</a>');
return $article_html;
}
$article_title = trim($element->find('h2', 0)->plaintext);
$article_uri = self::URI.($element->find('a', 0)->href);
$article_timestamp = strtotime($element->find('time.assetTime', 0)->plaintext);
$article_author = trim($element->find('a[rel=author]', 0)->plaintext);
$pageUrl = self::URI . (empty($this->getInput('topic')) ? '' : 'topics/' . $this->getInput('topic') . '/');
$html = getSimpleHTMLDOM($pageUrl) or returnServerError('Could not request CNET: ' . $pageUrl);
$limit = 0;
if (!empty($article_title) && !empty($article_uri) && strpos($article_uri, '/news/') !== false) {
foreach($html->find('div.assetBody') as $element) {
if($limit < 8) {
$article_title = trim($element->find('h2', 0)->plaintext);
$article_uri = self::URI . ($element->find('a', 0)->href);
$article_timestamp = strtotime($element->find('time.assetTime', 0)->plaintext);
$article_author = trim($element->find('a[rel=author]', 0)->plaintext);
$article_html = getSimpleHTMLDOM($article_uri) or returnServerError('Could not request CNET: '.$article_uri);
if(!empty($article_title) && !empty($article_uri) && strpos($article_uri, '/news/') !== false) {
$article_html = getSimpleHTMLDOM($article_uri)
or returnServerError('Could not request CNET: ' . $article_uri);
$article_content = trim(
cleanArticle(
extractFromDelimiters(
$article_html,
'<div class="articleContent',
'<footer>'
)
)
);
$article_content = trim(CleanArticle(ExtractFromDelimiters($article_html, '<div class="articleContent', '<footer>')));
$item = array();
$item['uri'] = $article_uri;
$item['title'] = $article_title;
$item['author'] = $article_author;
$item['timestamp'] = $article_timestamp;
$item['content'] = $article_content;
$this->items[] = $item;
$limit++;
}
}
}
}
$item = array();
$item['uri'] = $article_uri;
$item['title'] = $article_title;
$item['author'] = $article_author;
$item['timestamp'] = $article_timestamp;
$item['content'] = $article_content;
$this->items[] = $item;
$limit++;
}
}
}
}
public function getName(){
if(!is_null($this->getInput('topic'))) {
$topic = $this->getInput('topic');
return 'CNET News Bridge' . (empty($topic) ? '' : ' - ' . $topic);
}
public function getName() {
$topic=$this->getInput('topic');
return 'CNET News Bridge'.(empty($topic) ? '' : ' - '.$topic);
}
return parent::getName();
}
}

View File

@@ -1,35 +1,35 @@
<?php
class CastorusBridge extends BridgeAbstract {
const MAINTAINER = "logmanoriginal";
const NAME = "Castorus Bridge";
const MAINTAINER = 'logmanoriginal';
const NAME = 'Castorus Bridge';
const URI = 'http://www.castorus.com';
const CACHE_TIMEOUT = 600; // 10min
const DESCRIPTION = "Returns the latest changes";
const DESCRIPTION = 'Returns the latest changes';
const PARAMETERS = array(
'Get latest changes' => array(),
'Get latest changes via ZIP code' => array(
'zip'=>array(
'name'=>'ZIP code',
'type'=>'text',
'required'=>true,
'exampleValue'=>'74910, 74',
'title'=>'Insert ZIP code (complete or partial)'
)
),
'Get latest changes via city name' => array(
'city'=>array(
'name'=>'City name',
'type'=>'text',
'required'=>true,
'exampleValue'=>'Seyssel, Seys',
'title'=>'Insert city name (complete or partial)'
)
)
);
const PARAMETERS = array(
'Get latest changes' => array(),
'Get latest changes via ZIP code' => array(
'zip' => array(
'name' => 'ZIP code',
'type' => 'text',
'required' => true,
'exampleValue' => '74910, 74',
'title' => 'Insert ZIP code (complete or partial)'
)
),
'Get latest changes via city name' => array(
'city' => array(
'name' => 'City name',
'type' => 'text',
'required' => true,
'exampleValue' => 'Seyssel, Seys',
'title' => 'Insert city name (complete or partial)'
)
)
);
// Extracts the tile from an actitiy
private function ExtractActivityTitle($activity){
// Extracts the title from an actitiy
private function extractActivityTitle($activity){
$title = $activity->find('a', 0);
if(!$title)
@@ -39,7 +39,7 @@ class CastorusBridge extends BridgeAbstract {
}
// Extracts the url from an actitiy
private function ExtractActivityUrl($activity){
private function extractActivityUrl($activity){
$url = $activity->find('a', 0);
if(!$url)
@@ -49,7 +49,7 @@ class CastorusBridge extends BridgeAbstract {
}
// Extracts the time from an activity
private function ExtractActivityTime($activity){
private function extractActivityTime($activity){
// Unfortunately the time is part of the parent node,
// so we have to clear all child nodes first
$nodes = $activity->find('*');
@@ -57,7 +57,7 @@ class CastorusBridge extends BridgeAbstract {
if(!$nodes)
returnServerError('Cannot find nodes!');
foreach($nodes as $node){
foreach($nodes as $node) {
$node->outertext = '';
}
@@ -65,7 +65,7 @@ class CastorusBridge extends BridgeAbstract {
}
// Extracts the price change
private function ExtractActivityPrice($activity){
private function extractActivityPrice($activity){
$price = $activity->find('span', 1);
if(!$price)
@@ -75,8 +75,8 @@ class CastorusBridge extends BridgeAbstract {
}
public function collectData(){
$zip_filter = trim($this->getInput('zip'));
$city_filter = trim($this->getInput('city'));
$zip_filter = trim($this->getInput('zip'));
$city_filter = trim($this->getInput('city'));
$html = getSimpleHTMLDOM(self::URI);
@@ -88,20 +88,27 @@ class CastorusBridge extends BridgeAbstract {
if(!$activities)
returnServerError('Failed to find activities!');
foreach($activities as $activity){
foreach($activities as $activity) {
$item = array();
$item['title'] = $this->ExtractActivityTitle($activity);
$item['uri'] = $this->ExtractActivityUrl($activity);
$item['timestamp'] = $this->ExtractActivityTime($activity);
$item['content'] = '<a href="' . $item['uri'] . '">' . $item['title'] . '</a><br><p>'
. $this->ExtractActivityPrice($activity) . '</p>';
$item['title'] = $this->extractActivityTitle($activity);
$item['uri'] = $this->extractActivityUrl($activity);
$item['timestamp'] = $this->extractActivityTime($activity);
$item['content'] = '<a href="'
. $item['uri']
. '">'
. $item['title']
. '</a><br><p>'
. $this->extractActivityPrice($activity)
. '</p>';
if(isset($zip_filter) && !(substr($item['title'], 0, strlen($zip_filter)) === $zip_filter)){
if(isset($zip_filter)
&& !(substr($item['title'], 0, strlen($zip_filter)) === $zip_filter)) {
continue; // Skip this item
}
if(isset($city_filter) && !(substr($item['title'], strpos($item['title'], ' ') + 1, strlen($city_filter)) === $city_filter)){
if(isset($city_filter)
&& !(substr($item['title'], strpos($item['title'], ' ') + 1, strlen($city_filter)) === $city_filter)) {
continue; // Skip this item
}

View File

@@ -0,0 +1,25 @@
<?php
class ChristianDailyReporterBridge extends BridgeAbstract {
const MAINTAINER = 'rogerdc';
const NAME = 'Christian Daily Reporter Unofficial RSS';
const URI = 'https://www.christiandailyreporter.com/';
const DESCRIPTION = 'The Unofficial Christian Daily Reporter RSS';
// const CACHE_TIMEOUT = 86400; // 1 day
public function collectData() {
$uri = 'https://www.christiandailyreporter.com/';
$html = getSimpleHTMLDOM($uri)
or returnServerError('Could not request Christian Daily Reporter.');
foreach($html->find('div.top p a,div.column p a') as $element) {
$item = array();
// Title
$item['title'] = $element->innertext;
// URL
$item['uri'] = $element->href;
$this->items[] = $item;
}
}
}

View File

@@ -1,11 +1,11 @@
<?php
class CollegeDeFranceBridge extends BridgeAbstract{
class CollegeDeFranceBridge extends BridgeAbstract {
const MAINTAINER = "pit-fgfjiudghdf";
const NAME = "CollegeDeFrance";
const URI = "http://www.college-de-france.fr/";
const MAINTAINER = 'pit-fgfjiudghdf';
const NAME = 'CollegeDeFrance';
const URI = 'http://www.college-de-france.fr/';
const CACHE_TIMEOUT = 10800; // 3h
const DESCRIPTION = "Returns the latest audio and video from CollegeDeFrance";
const DESCRIPTION = 'Returns the latest audio and video from CollegeDeFrance';
public function collectData(){
$months = array(
@@ -22,47 +22,61 @@ class CollegeDeFranceBridge extends BridgeAbstract{
'11' => 'nov.',
'12' => 'déc.'
);
// The "API" used by the site returns a list of partial HTML in this form
/* <li>
* <a href="/site/thomas-romer/guestlecturer-2016-04-15-14h30.htm" data-target="after">
* <span class="date"><span class="list-icon list-icon-video"></span><span class="list-icon list-icon-audio"></span>15 avr. 2016</span>
* <span class="date"><span class="list-icon list-icon-video"></span>
* <span class="list-icon list-icon-audio"></span>15 avr. 2016</span>
* <span class="lecturer">Christopher Hays</span>
* <span class='title'>Imagery of Divine Suckling in the Hebrew Bible and the Ancient Near East</span>
* </a>
* </li>
*/
$html = getSimpleHTMLDOM(self::URI.'components/search-audiovideo.jsp?fulltext=&siteid=1156951719600&lang=FR&type=all')
or returnServerError('Could not request CollegeDeFrance.');
$html = getSimpleHTMLDOM(self::URI
. 'components/search-audiovideo.jsp?fulltext=&siteid=1156951719600&lang=FR&type=all')
or returnServerError('Could not request CollegeDeFrance.');
foreach($html->find('a[data-target]') as $element) {
$item = array();
$item['title'] = $element->find('.title', 0)->plaintext;
// Most relative URLs contains an hour in addition to the date, so let's use it
// <a href="/site/yann-lecun/course-2016-04-08-11h00.htm" data-target="after">
//
// Sometimes there's an __1, perhaps it signifies an update "/site/patrick-boucheron/seminar-2016-05-03-18h00__1.htm"
// Sometimes there's an __1, perhaps it signifies an update
// "/site/patrick-boucheron/seminar-2016-05-03-18h00__1.htm"
//
// But unfortunately some don't have any hours info
// <a href="/site/institut-physique/The-Mysteries-of-Decoherence-Sebastien-Gleyzes-[Video-3-35].htm" data-target="after">
// <a href="/site/institut-physique/
// The-Mysteries-of-Decoherence-Sebastien-Gleyzes-[Video-3-35].htm" data-target="after">
$timezone = new DateTimeZone('Europe/Paris');
// strpos($element->href, '201') will break in 2020 but it'll probably break prior to then due to site changes anyway
$d = DateTime::createFromFormat(
'!Y-m-d-H\hi',
substr($element->href, strpos($element->href, '201'), 16),
$timezone
);
if(!$d){
$d=DateTime::createFromFormat(
'!d m Y',
trim(str_replace(
array_values($months),
array_keys($months),
$element->find('.date', 0)->plaintext
)),
$timezone
);
}
$item['timestamp'] = $d->format('U');
$item['content'] = $element->find('.lecturer', 0)->innertext . ' - ' . $element->find('.title', 0)->innertext;
// strpos($element->href, '201') will break in 2020 but it'll
// probably break prior to then due to site changes anyway
$d = DateTime::createFromFormat(
'!Y-m-d-H\hi',
substr($element->href, strpos($element->href, '201'), 16),
$timezone
);
if(!$d) {
$d = DateTime::createFromFormat(
'!d m Y',
trim(str_replace(
array_values($months),
array_keys($months),
$element->find('.date', 0)->plaintext
)),
$timezone
);
}
$item['timestamp'] = $d->format('U');
$item['content'] = $element->find('.lecturer', 0)->innertext
. ' - '
. $element->find('.title', 0)->innertext;
$item['uri'] = self::URI . $element->href;
$this->items[] = $item;
}

View File

@@ -1,10 +1,10 @@
<?php
class CommonDreamsBridge extends FeedExpander {
const MAINTAINER = "nyutag";
const NAME = "CommonDreams Bridge";
const URI = "http://www.commondreams.org/";
const DESCRIPTION = "Returns the newest articles.";
const MAINTAINER = 'nyutag';
const NAME = 'CommonDreams Bridge';
const URI = 'http://www.commondreams.org/';
const DESCRIPTION = 'Returns the newest articles.';
public function collectData(){
$this->collectExpandableDatas('http://www.commondreams.org/rss.xml', 10);
@@ -12,11 +12,11 @@ class CommonDreamsBridge extends FeedExpander {
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
$item['content'] = $this->CommonDreamsExtractContent($item['uri']);
$item['content'] = $this->extractContent($item['uri']);
return $item;
}
private function CommonDreamsExtractContent($url) {
private function extractContent($url){
$html3 = getSimpleHTMLDOMCached($url);
$text = $html3->find('div[class=field--type-text-with-summary]', 0)->innertext;
$html3->clear();

View File

@@ -0,0 +1,93 @@
<?php
class ContainerLinuxReleasesBridge extends BridgeAbstract {
const MAINTAINER = 'captn3m0';
const NAME = 'Core OS Container Linux Releases Bridge';
const URI = 'https://coreos.com/releases/';
const DESCRIPTION = 'Returns the releases notes for Container Linux';
const STABLE = 'stable';
const BETA = 'beta';
const ALPHA = 'alpha';
const PARAMETERS = [
[
'channel' => [
'name' => 'Release Channel',
'type' => 'list',
'required' => true,
'defaultValue' => self::STABLE,
'values' => [
'Stable' => self::STABLE,
'Beta' => self::BETA,
'Alpha' => self::ALPHA,
],
]
]
];
public function getReleaseFeed($jsonUrl) {
$json = getContents($jsonUrl)
or returnServerError('Could not request Core OS Website.');
return json_decode($json, true);
}
public function collectData() {
$data = $this->getReleaseFeed($this->getJsonUri());
foreach ($data as $releaseVersion => $release) {
$item = [];
$item['uri'] = "https://coreos.com/releases/#$releaseVersion";
$item['title'] = $releaseVersion;
$content = $release['release_notes'];
$content .= <<<EOT
Major Software:
* Kernel: {$release['major_software']['kernel'][0]}
* Docker: {$release['major_software']['docker'][0]}
* etcd: {$release['major_software']['etcd'][0]}
EOT;
$item['timestamp'] = strtotime($release['release_date']);
// Based on https://gist.github.com/jbroadway/2836900
// Links
$regex = '/\[([^\[]+)\]\(([^\)]+)\)/';
$replacement = '<a href=\'\2\'>\1</a>';
$item['content'] = preg_replace($regex, $replacement, $content);
// Headings
$regex = '/^(.*)\:\s?$/m';
$replacement = '<h3>\1</h3>';
$item['content'] = preg_replace($regex, $replacement, $item['content']);
// Lists
$regex = '/\n\s*[\*|\-](.*)/';
$item['content'] = preg_replace_callback ($regex, function($regs) {
$item = $regs[1];
return sprintf ('<ul><li>%s</li></ul>', trim ($item));
}, $item['content']);
$this->items[] = $item;
}
}
private function getJsonUri() {
$channel = $this->getInput('channel');
return "https://coreos.com/releases/releases-$channel.json";
}
public function getURI() {
return self::URI;
}
public function getName(){
if(!is_null($this->getInput('channel'))) {
return 'Container Linux Releases: ' . $this->getInput('channel') . ' Channel';
}
return parent::getName();
}
}

View File

@@ -1,39 +1,35 @@
<?php
class CopieDoubleBridge extends BridgeAbstract{
class CopieDoubleBridge extends BridgeAbstract {
const MAINTAINER = "superbaillot.net";
const NAME = "CopieDouble";
const URI = "http://www.copie-double.com/";
const CACHE_TIMEOUT = 14400; // 4h
const DESCRIPTION = "CopieDouble";
const MAINTAINER = 'superbaillot.net';
const NAME = 'CopieDouble';
const URI = 'http://www.copie-double.com/';
const CACHE_TIMEOUT = 14400; // 4h
const DESCRIPTION = 'CopieDouble';
public function collectData(){
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not request CopieDouble.');
$table = $html->find('table table', 2);
public function collectData(){
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not request CopieDouble.');
foreach($table->find('tr') as $element)
{
$td = $element->find('td', 0);
if($td->class == "couleur_1")
{
$item = array();
$table = $html->find('table table', 2);
$title = $td->innertext;
$pos = strpos($title, "<a");
$title = substr($title, 0, $pos);
$item['title'] = $title;
}
elseif(strpos($element->innertext, "/images/suivant.gif") === false)
{
$a=$element->find("a", 0);
$item['uri'] = self::URI . $a->href;
foreach($table->find('tr') as $element) {
$td = $element->find('td', 0);
$content = str_replace('src="/', 'src="/'.self::URI,$element->find("td", 0)->innertext);
$content = str_replace('href="/', 'href="'.self::URI,$content);
$item['content'] = $content;
$this->items[] = $item;
}
}
}
if($td->class === 'couleur_1') {
$item = array();
$title = $td->innertext;
$pos = strpos($title, '<a');
$title = substr($title, 0, $pos);
$item['title'] = $title;
} elseif(strpos($element->innertext, '/images/suivant.gif') === false) {
$a = $element->find('a', 0);
$item['uri'] = self::URI . $a->href;
$content = str_replace('src="/', 'src="/' . self::URI, $element->find('td', 0)->innertext);
$content = str_replace('href="/', 'href="' . self::URI, $content);
$item['content'] = $content;
$this->items[] = $item;
}
}
}
}

View File

@@ -1,61 +1,55 @@
<?php
class CourrierInternationalBridge extends BridgeAbstract{
class CourrierInternationalBridge extends BridgeAbstract {
const MAINTAINER = "teromene";
const NAME = "Courrier International Bridge";
const URI = "http://CourrierInternational.com/";
const CACHE_TIMEOUT = 300; // 5 min
const DESCRIPTION = "Courrier International bridge";
const MAINTAINER = 'teromene';
const NAME = 'Courrier International Bridge';
const URI = 'http://CourrierInternational.com/';
const CACHE_TIMEOUT = 300; // 5 min
const DESCRIPTION = 'Courrier International bridge';
public function collectData(){
public function collectData(){
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Error.');
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Error.');
$element = $html->find('article');
$article_count = 1;
$element = $html->find("article");
foreach($element as $article) {
$item = array();
$article_count = 1;
$item['uri'] = $article->parent->getAttribute('href');
foreach($element as $article) {
if(strpos($item['uri'], 'http') === false) {
$item['uri'] = self::URI . $item['uri'];
}
$item = array();
$page = getSimpleHTMLDOMCached($item['uri']);
$item['uri'] = $article->parent->getAttribute("href");
$content = $page->find('.article-text', 0);
if(strpos($item['uri'], "http") === FALSE) {
$item['uri'] = self::URI.$item['uri'];
}
if(!$content) {
$content = $page->find('.depeche-text', 0);
}
$page = getSimpleHTMLDOMCached($item['uri']);
$item['content'] = sanitize($content);
$item['title'] = strip_tags($article->find('.title', 0));
$content = $page->find('.article-text',0);
if(!$content){
$content = $page->find('.depeche-text',0);
}
$dateTime = date_parse($page->find('time', 0));
$item['content'] = sanitize($content);
$item['title'] = strip_tags($article->find(".title",0));
$item['timestamp'] = mktime(
$dateTime['hour'],
$dateTime['minute'],
$dateTime['second'],
$dateTime['month'],
$dateTime['day'],
$dateTime['year']
);
$dateTime = date_parse($page->find("time",0));
$this->items[] = $item;
$article_count ++;
$item['timestamp'] = mktime(
$dateTime['hour'],
$dateTime['minute'],
$dateTime['second'],
$dateTime['month'],
$dateTime['day'],
$dateTime['year']
);
$this->items[] = $item;
$article_count ++;
if($article_count > 5) break;
}
}
if($article_count > 5)
break;
}
}
}
?>

View File

@@ -1,53 +1,74 @@
<?php
class CpasbienBridge extends BridgeAbstract {
const MAINTAINER = "lagaisse";
const NAME = "Cpasbien Bridge";
const URI = "http://www.cpasbien.io";
const CACHE_TIMEOUT = 86400; // 24h
const DESCRIPTION = "Returns latest torrents from a request query";
const MAINTAINER = 'lagaisse';
const NAME = 'Cpasbien Bridge';
const URI = 'http://www.cpasbien.cm';
const CACHE_TIMEOUT = 86400; // 24h
const DESCRIPTION = 'Returns latest torrents from a request query';
const PARAMETERS = array( array(
'q'=>array(
'name'=>'Search',
'required'=>true,
'title'=>'Type your search'
)
));
const PARAMETERS = array( array(
'q' => array(
'name' => 'Search',
'required' => true,
'title' => 'Type your search'
)
));
public function collectData(){
$request = str_replace(" ","-",trim($this->getInput('q')));
$html = getSimpleHTMLDOM(self::URI.'/recherche/'.urlencode($request).'.html')
or returnServerError('No results for this query.');
public function collectData(){
$request = str_replace(' ', '-', trim($this->getInput('q')));
$html = getSimpleHTMLDOM(self::URI . '/recherche/' . urlencode($request) . '.html')
or returnServerError('No results for this query.');
foreach ($html->find('#gauche',0)->find('div') as $episode) {
if ($episode->getAttribute('class')=='ligne0' ||
$episode->getAttribute('class')=='ligne1')
{
$htmlepisode=getSimpleHTMLDOMCached($episode->find('a', 0)->getAttribute('href'));
foreach($html->find('#gauche', 0)->find('div') as $episode) {
if($episode->getAttribute('class') == 'ligne0'
|| $episode->getAttribute('class') == 'ligne1') {
$item = array();
$item['author'] = $episode->find('a', 0)->text();
$item['title'] = $episode->find('a', 0)->text();
$textefiche=$htmlepisode->find('#textefiche', 0)->find('p',1);
if (isset($textefiche)) {
$item['content'] = $textefiche->text();
} else {
$p=$htmlepisode->find('#textefiche',0)->find('p');
if(!empty($p)){
$item['content'] = $htmlepisode->find('#textefiche', 0)->find('p',0)->text();
}
}
$urlepisode = $episode->find('a', 0)->getAttribute('href');
$htmlepisode = getSimpleHTMLDOMCached($urlepisode, 86400 * 366 * 30);
$item['id'] = $episode->find('a', 0)->getAttribute('href');
$item['uri'] = self::URI . $htmlepisode->find('#telecharger',0)->getAttribute('href');
$this->items[] = $item;
}
}
}
$item = array();
$item['author'] = $episode->find('a', 0)->text();
$item['title'] = $episode->find('a', 0)->text();
$item['pubdate'] = $this->getCachedDate($urlepisode);
$textefiche = $htmlepisode->find('#textefiche', 0)->find('p', 1);
if(isset($textefiche)) {
$item['content'] = $textefiche->text();
} else {
$p = $htmlepisode->find('#textefiche', 0)->find('p');
if(!empty($p)) {
$item['content'] = $htmlepisode->find('#textefiche', 0)->find('p', 0)->text();
}
}
public function getName(){
return $this->getInput('q').' : '.self::NAME;
}
$item['id'] = $episode->find('a', 0)->getAttribute('href');
$item['uri'] = self::URI . $htmlepisode->find('#telecharger', 0)->getAttribute('href');
$this->items[] = $item;
}
}
}
public function getName(){
if(!is_null($this->getInput('q'))) {
return $this->getInput('q') . ' : ' . self::NAME;
}
return parent::getName();
}
private function getCachedDate($url){
debugMessage('getting pubdate from url ' . $url . '');
// Initialize cache
$cache = Cache::create('FileCache');
$cache->setPath(CACHE_DIR . '/pages');
$params = [$url];
$cache->setParameters($params);
// Get cachefile timestamp
$time = $cache->getTime();
return ($time !== false ? $time : time());
}
}

View File

@@ -1,39 +1,45 @@
<?php
class CryptomeBridge extends BridgeAbstract{
class CryptomeBridge extends BridgeAbstract {
const MAINTAINER = "BoboTiG";
const NAME = "Cryptome";
const URI = "https://cryptome.org/";
const CACHE_TIMEOUT = 21600; //6h
const DESCRIPTION = "Returns the N most recent documents.";
const MAINTAINER = 'BoboTiG';
const NAME = 'Cryptome';
const URI = 'https://cryptome.org/';
const CACHE_TIMEOUT = 21600; //6h
const DESCRIPTION = 'Returns the N most recent documents.';
const PARAMETERS = array( array(
'n'=>array(
'name'=>'number of elements',
'type'=>'number',
'defaultValue'=>20,
'exampleValue'=>10
)
));
const PARAMETERS = array( array(
'n' => array(
'name' => 'number of elements',
'type' => 'number',
'defaultValue' => 20,
'exampleValue' => 10
)
));
public function collectData(){
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not request Cryptome.');
$number=$this->getInput('n');
if (!empty($number)) { /* number of documents */
$num = min($number, 20);
}
public function collectData(){
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not request Cryptome.');
$number = $this->getInput('n');
foreach($html->find('pre') as $element) {
for ( $i = 0; $i < $num; ++$i ) {
$item = array();
$item['uri'] = self::URI.substr($element->find('a', $i)->href, 20);
$item['title'] = substr($element->find('b', $i)->plaintext, 22);
$item['content'] = preg_replace('#http://cryptome.org/#', self::URI, $element->find('b', $i)->innertext);
$this->items[] = $item;
}
break;
}
}
/* number of documents */
if(!empty($number)) {
$num = min($number, 20);
}
foreach($html->find('pre') as $element) {
for($i = 0; $i < $num; ++$i) {
$item = array();
$item['uri'] = self::URI . substr($element->find('a', $i)->href, 20);
$item['title'] = substr($element->find('b', $i)->plaintext, 22);
$item['content'] = preg_replace(
'#http://cryptome.org/#',
self::URI,
$element->find('b', $i)->innertext
);
$this->items[] = $item;
}
break;
}
}
}

View File

@@ -1,114 +1,123 @@
<?php
class DailymotionBridge extends BridgeAbstract{
class DailymotionBridge extends BridgeAbstract {
const MAINTAINER = "mitsukarenai";
const NAME = "Dailymotion Bridge";
const URI = "https://www.dailymotion.com/";
const CACHE_TIMEOUT = 10800; // 3h
const DESCRIPTION = "Returns the 5 newest videos by username/playlist or search";
const MAINTAINER = 'mitsukarenai';
const NAME = 'Dailymotion Bridge';
const URI = 'https://www.dailymotion.com/';
const CACHE_TIMEOUT = 10800; // 3h
const DESCRIPTION = 'Returns the 5 newest videos by username/playlist or search';
const PARAMETERS = array (
'By username' => array(
'u'=>array(
'name'=>'username',
'required'=>true
)
),
const PARAMETERS = array (
'By username' => array(
'u' => array(
'name' => 'username',
'required' => true
)
),
'By playlist id' => array(
'p' => array(
'name' => 'playlist id',
'required' => true
)
),
'From search results' => array(
's' => array(
'name' => 'Search keyword',
'required' => true
),
'pa' => array(
'name' => 'Page',
'type' => 'number'
)
)
);
'By playlist id' => array(
'p'=>array(
'name'=>'playlist id',
'required'=>true
)
),
protected function getMetadata($id){
$metadata = array();
$html2 = getSimpleHTMLDOM(self::URI . 'video/' . $id);
if(!$html2) {
return $metadata;
}
'From search results' => array(
's'=>array(
'name'=>'Search keyword',
'required'=>true
),
'pa'=>array(
'name'=>'Page',
'type'=>'number'
)
)
);
$metadata['title'] = $html2->find('meta[property=og:title]', 0)->getAttribute('content');
$metadata['timestamp'] = strtotime(
$html2->find('meta[property=video:release_date]', 0)->getAttribute('content')
);
$metadata['thumbnailUri'] = $html2->find('meta[property=og:image]', 0)->getAttribute('content');
$metadata['uri'] = $html2->find('meta[property=og:url]', 0)->getAttribute('content');
return $metadata;
}
function getMetadata($id) {
$metadata=array();
$html2 = getSimpleHTMLDOM(self::URI.'video/'.$id);
if(!$html2){
return $metadata;
}
public function collectData(){
$html = '';
$limit = 5;
$count = 0;
$metadata['title'] = $html2->find('meta[property=og:title]', 0)->getAttribute('content');
$metadata['timestamp'] = strtotime($html2->find('meta[property=video:release_date]', 0)->getAttribute('content') );
$metadata['thumbnailUri'] = $html2->find('meta[property=og:image]', 0)->getAttribute('content');
$metadata['uri'] = $html2->find('meta[property=og:url]', 0)->getAttribute('content');
return $metadata;
}
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request Dailymotion.');
public function collectData(){
$html = '';
$limit = 5;
$count = 0;
foreach($html->find('div.media a.preview_link') as $element) {
if($count < $limit) {
$item = array();
$item['id'] = str_replace('/video/', '', strtok($element->href, '_'));
$metadata = $this->getMetadata($item['id']);
if(empty($metadata)) {
continue;
}
$item['uri'] = $metadata['uri'];
$item['title'] = $metadata['title'];
$item['timestamp'] = $metadata['timestamp'];
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request Dailymotion.');
$item['content'] = '<a href="'
. $item['uri']
. '"><img src="'
. $metadata['thumbnailUri']
. '" /></a><br><a href="'
. $item['uri']
. '">'
. $item['title']
. '</a>';
foreach($html->find('div.media a.preview_link') as $element) {
if($count < $limit) {
$item = array();
$item['id'] = str_replace('/video/', '', strtok($element->href, '_'));
$metadata = $this->getMetadata($item['id']);
if(empty($metadata)){
continue;
}
$item['uri'] = $metadata['uri'];
$item['title'] = $metadata['title'];
$item['timestamp'] = $metadata['timestamp'];
$item['content'] = '<a href="' . $item['uri'] . '"><img src="' . $metadata['thumbnailUri'] . '" /></a><br><a href="' . $item['uri'] . '">' . $item['title'] . '</a>';
$this->items[] = $item;
$count++;
}
}
}
$this->items[] = $item;
$count++;
}
}
}
public function getName(){
switch($this->queriedContext){
case 'By username':
$specific=$this->getInput('u');
break;
case 'By playlist id':
$specific=strtok($this->getInput('p'), '_');
break;
case 'From search results':
$specific=$this->getInput('s');
break;
}
public function getName(){
switch($this->queriedContext) {
case 'By username':
$specific = $this->getInput('u');
break;
case 'By playlist id':
$specific = strtok($this->getInput('p'), '_');
break;
case 'From search results':
$specific = $this->getInput('s');
break;
default: return parent::getName();
}
return $specific.' : Dailymotion Bridge';
}
return $specific . ' : Dailymotion Bridge';
}
public function getURI(){
$uri=self::URI;
switch($this->queriedContext){
case 'By username':
$uri.='user/'
.urlencode($this->getInput('u')).'/1';
break;
case 'By playlist id':
$uri.='playlist/'
.urlencode(strtok($this->getInput('p'), '_'));
break;
case 'From search results':
$uri.='search/'
.urlencode($this->getInput('s'));
if($this->getInput('pa')){
$uri.='/'.$this->getInput('pa');
}
break;
}
return $uri;
}
public function getURI(){
$uri = self::URI;
switch($this->queriedContext) {
case 'By username':
$uri .= 'user/' . urlencode($this->getInput('u')) . '/1';
break;
case 'By playlist id':
$uri .= 'playlist/' . urlencode(strtok($this->getInput('p'), '_'));
break;
case 'From search results':
$uri .= 'search/' . urlencode($this->getInput('s'));
if($this->getInput('pa')) {
$uri .= '/' . $this->getInput('pa');
}
break;
default: return parent::getURI();
}
return $uri;
}
}

View File

@@ -1,51 +1,67 @@
<?php
class DanbooruBridge extends BridgeAbstract{
class DanbooruBridge extends BridgeAbstract {
const MAINTAINER = "mitsukarenai";
const NAME = "Danbooru";
const URI = "http://donmai.us/";
const CACHE_TIMEOUT = 1800; // 30min
const DESCRIPTION = "Returns images from given page";
const MAINTAINER = 'mitsukarenai';
const NAME = 'Danbooru';
const URI = 'http://donmai.us/';
const CACHE_TIMEOUT = 1800; // 30min
const DESCRIPTION = 'Returns images from given page';
const PARAMETERS = array(
'global'=>array(
'p'=>array(
'name'=>'page',
'defaultValue'=>1,
'type'=>'number'
),
't'=>array('name'=>'tags')
),
0=>array()
);
const PARAMETERS = array(
'global' => array(
'p' => array(
'name' => 'page',
'defaultValue' => 1,
'type' => 'number'
),
't' => array(
'name' => 'tags'
)
),
0 => array()
);
const PATHTODATA='article';
const IDATTRIBUTE='data-id';
const PATHTODATA = 'article';
const IDATTRIBUTE = 'data-id';
const TAGATTRIBUTE = 'alt';
protected function getFullURI(){
return $this->getURI().'posts?'
.'&page='.$this->getInput('p')
.'&tags='.urlencode($this->getInput('t'));
}
protected function getFullURI(){
return $this->getURI()
. 'posts?&page=' . $this->getInput('p')
. '&tags=' . urlencode($this->getInput('t'));
}
protected function getItemFromElement($element){
$item = array();
$item['uri'] = $this->getURI().$element->find('a', 0)->href;
$item['postid'] = (int)preg_replace("/[^0-9]/",'', $element->getAttribute(static::IDATTRIBUTE));
$item['timestamp'] = time();
$thumbnailUri = $this->getURI().$element->find('img', 0)->src;
$item['tags'] = $element->find('img', 0)->getAttribute('alt');
$item['title'] = $this->getName().' | '.$item['postid'];
$item['content'] = '<a href="' . $item['uri'] . '"><img src="' . $thumbnailUri . '" /></a><br>Tags: '.$item['tags'];
return $item;
}
protected function getTags($element){
return $element->find('img', 0)->getAttribute(static::TAGATTRIBUTE);
}
public function collectData(){
$html = getSimpleHTMLDOM($this->getFullURI())
or returnServerError('Could not request '.$this->getName());
protected function getItemFromElement($element){
// Fix links
defaultLinkTo($element, $this->getURI());
foreach($html->find(static::PATHTODATA) as $element) {
$this->items[] = $this->getItemFromElement($element);
}
}
$item = array();
$item['uri'] = $element->find('a', 0)->href;
$item['postid'] = (int)preg_replace('/[^0-9]/', '', $element->getAttribute(static::IDATTRIBUTE));
$item['timestamp'] = time();
$thumbnailUri = $element->find('img', 0)->src;
$item['tags'] = $this->getTags($element);
$item['title'] = $this->getName() . ' | ' . $item['postid'];
$item['content'] = '<a href="'
. $item['uri']
. '"><img src="'
. $thumbnailUri
. '" /></a><br>Tags: '
. $item['tags'];
return $item;
}
public function collectData(){
$html = getSimpleHTMLDOM($this->getFullURI())
or returnServerError('Could not request ' . $this->getName());
foreach($html->find(static::PATHTODATA) as $element) {
$this->items[] = $this->getItemFromElement($element);
}
}
}

View File

@@ -1,23 +1,28 @@
<?php
class DansTonChatBridge extends BridgeAbstract{
class DansTonChatBridge extends BridgeAbstract {
const MAINTAINER = "Astalaseven";
const NAME = "DansTonChat Bridge";
const URI = "http://danstonchat.com/";
const MAINTAINER = 'Astalaseven';
const NAME = 'DansTonChat Bridge';
const URI = 'https://danstonchat.com/';
const CACHE_TIMEOUT = 21600; //6h
const DESCRIPTION = "Returns latest quotes from DansTonChat.";
const DESCRIPTION = 'Returns latest quotes from DansTonChat.';
public function collectData(){
public function collectData(){
$html = getSimpleHTMLDOM(self::URI.'latest.html')
or returnServerError('Could not request DansTonChat.');
$html = getSimpleHTMLDOM(self::URI . 'latest.html')
or returnServerError('Could not request DansTonChat.');
foreach($html->find('div.item') as $element) {
$item = array();
$item['uri'] = $element->find('a', 0)->href;
$item['title'] = 'DansTonChat '.$element->find('a', 1)->plaintext;
$item['content'] = $element->find('a', 0)->innertext;
$this->items[] = $item;
}
}
foreach($html->find('div.item') as $element) {
$item = array();
$item['uri'] = $element->find('a', 0)->href;
$titleContent = $element->find('h3 a', 0);
if($titleContent) {
$item['title'] = 'DansTonChat ' . html_entity_decode($titleContent->plaintext, ENT_QUOTES);
} else {
$item['title'] = 'DansTonChat';
}
$item['content'] = $element->find('div.item-content a', 0)->innertext;
$this->items[] = $item;
}
}
}

View File

@@ -1,57 +1,56 @@
<?php
class DauphineLibereBridge extends FeedExpander {
const MAINTAINER = "qwertygc";
const NAME = "Dauphine Bridge";
const URI = "http://www.ledauphine.com/";
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = "Returns the newest articles.";
const MAINTAINER = 'qwertygc';
const NAME = 'Dauphine Bridge';
const URI = 'http://www.ledauphine.com/';
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = 'Returns the newest articles.';
const PARAMETERS = array( array(
'u'=>array(
'name'=>'Catégorie de l\'article',
'type'=>'list',
'values'=>array(
'À la une'=>'',
'France Monde'=>'france-monde',
'Faits Divers'=>'faits-divers',
'Économie et Finance'=>'economie-et-finance',
'Politique'=>'politique',
'Sport'=>'sport',
'Ain'=>'ain',
'Alpes-de-Haute-Provence'=>'haute-provence',
'Hautes-Alpes'=>'hautes-alpes',
'Ardèche'=>'ardeche',
'Drôme'=>'drome',
'Isère Sud'=>'isere-sud',
'Savoie'=>'savoie',
'Haute-Savoie'=>'haute-savoie',
'Vaucluse'=>'vaucluse'
)
)
));
const PARAMETERS = array( array(
'u' => array(
'name' => 'Catégorie de l\'article',
'type' => 'list',
'values' => array(
'À la une' => '',
'France Monde' => 'france-monde',
'Faits Divers' => 'faits-divers',
'Économie et Finance' => 'economie-et-finance',
'Politique' => 'politique',
'Sport' => 'sport',
'Ain' => 'ain',
'Alpes-de-Haute-Provence' => 'haute-provence',
'Hautes-Alpes' => 'hautes-alpes',
'Ardèche' => 'ardeche',
'Drôme' => 'drome',
'Isère Sud' => 'isere-sud',
'Savoie' => 'savoie',
'Haute-Savoie' => 'haute-savoie',
'Vaucluse' => 'vaucluse'
)
)
));
public function collectData(){
$url = self::URI . 'rss';
public function collectData(){
$url = self::URI . 'rss';
if (empty($this->getInput('u'))) {
$url = self::URI . $this->getInput('u') . '/rss';
}
if(empty($this->getInput('u'))) {
$url = self::URI . $this->getInput('u') . '/rss';
}
$this->collectExpandableDatas($url, 10);
}
$this->collectExpandableDatas($url, 10);
}
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
$item['content'] = $this->ExtractContent($item['uri']);
return $item;
}
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
$item['content'] = $this->extractContent($item['uri']);
return $item;
}
private function ExtractContent($url) {
$html2 = getSimpleHTMLDOMCached($url);
$text = $html2->find('div.column', 0)->innertext;
$text = preg_replace('@<script[^>]*?>.*?</script>@si', '', $text);
return $text;
}
private function extractContent($url){
$html2 = getSimpleHTMLDOMCached($url);
$text = $html2->find('div.column', 0)->innertext;
$text = preg_replace('@<script[^>]*?>.*?</script>@si', '', $text);
return $text;
}
}
?>

589
bridges/DealabsBridge.php Normal file
View File

@@ -0,0 +1,589 @@
<?php
class DealabsBridge extends PepperBridgeAbstract {
const NAME = 'Dealabs Bridge';
const URI = 'https://www.dealabs.com/';
const DESCRIPTION = 'Affiche les Deals de Dealabs';
const MAINTAINER = 'sysadminstory';
const PARAMETERS = array(
'Recherche par Mot(s) clé(s)' => array (
'q' => array(
'name' => 'Mot(s) clé(s)',
'type' => 'text',
'required' => true
),
'hide_expired' => array(
'name' => 'Masquer les éléments expirés',
'type' => 'checkbox',
'required' => 'true'
),
'hide_local' => array(
'name' => 'Masquer les deals locaux',
'type' => 'checkbox',
'title' => 'Masquer les deals en magasins physiques',
'required' => 'true'
),
'priceFrom' => array(
'name' => 'Prix minimum',
'type' => 'text',
'title' => 'Prix mnimum en euros',
'required' => 'false',
'defaultValue' => ''
),
'priceTo' => array(
'name' => 'Prix maximum',
'type' => 'text',
'title' => 'Prix maximum en euros',
'required' => 'false',
'defaultValue' => ''
),
),
'Deals par groupe' => array(
'group' => array(
'name' => 'Groupe',
'type' => 'list',
'required' => 'true',
'title' => 'Groupe dont il faut afficher les deals',
'values' => array(
'Accessoires & gadgets' => 'accessoires-gadgets',
'Alimentation & boissons' => 'alimentation-boissons',
'Animaux' => 'animaux',
'Applis & logiciels' => 'applis-logiciels',
'Consoles & jeux vidéo' => 'consoles-jeux-video',
'Culture & divertissement' => 'culture-divertissement',
'Gratuit' => 'gratuit',
'Image, son & vidéo' => 'image-son-video',
'Informatique' => 'informatique',
'Jeux & jouets' => 'jeux-jouets',
'Maison & jardin' => 'maison-jardin',
'Mode & accessoires' => 'mode-accessoires',
'Santé & cosmétiques' => 'hygiene-sante-cosmetiques',
'Services divers' => 'services-divers',
'Sports & plein air' => 'sports-plein-air',
'Téléphonie' => 'telephonie',
'Voyages & sorties' => 'voyages-sorties-restaurants',
)
),
'order' => array(
'name' => 'Trier par',
'type' => 'list',
'required' => 'true',
'title' => 'Ordre de tri des deals',
'values' => array(
'Du deal le plus Hot au moins Hot' => '',
'Du deal le plus récent au plus ancien' => '-nouveaux',
'Du deal le plus commentés au moins commentés' => '-commentes'
)
)
)
);
public $lang = array(
'bridge-uri' => SELF::URI,
'bridge-name' => SELF::NAME,
'context-keyword' => 'Recherche par Mot(s) clé(s)',
'context-group' => 'Deals par groupe',
'uri-group' => '/groupe/',
'request-error' => 'Could not request Dealabs',
'no-results' => 'Il n&#039;y a rien à afficher pour le moment :(',
'relative-date-indicator' => array(
'il y a',
),
'price' => 'Prix',
'shipping' => 'Livraison',
'origin' => 'Origine',
'discount' => 'Réduction',
'title-keyword' => 'Recherche',
'title-group' => 'Groupe',
'local-months' => array(
'janvier',
'février',
'mars',
'avril',
'mai',
'juin',
'juillet',
'août',
'septembre',
'octobre',
'novembre',
'décembre'
),
'local-time-relative' => array(
'il y a ',
'min',
'h',
'jour',
'jours',
'mois',
'ans',
'et '
),
'date-prefixes' => array(
'Actualisé ',
),
'relative-date-alt-prefixes' => array(
'Actualisé ',
),
'relative-date-ignore-suffix' => array(
),
'localdeal' => array(
'Local',
'Pays d\'expédition'
),
);
}
class PepperBridgeAbstract extends BridgeAbstract {
const CACHE_TIMEOUT = 3600;
public function collectData(){
switch($this->queriedContext) {
case $this->i8n('context-keyword'):
return $this->collectDataKeywords();
break;
case $this->i8n('context-group'):
return $this->collectDataGroup();
break;
}
}
/**
* Get the Deal data from the choosen group in the choosed order
*/
public function collectDataGroup()
{
$group = $this->getInput('group');
$order = $this->getInput('order');
$url = $this->i8n('bridge-uri')
. $this->i8n('uri-group') . $group . $order;
$this->collectDeals($url);
}
/**
* Get the Deal data from the choosen keywords and parameters
*/
public function collectDataKeywords()
{
$q = $this->getInput('q');
$hide_expired = $this->getInput('hide_expired');
$hide_local = $this->getInput('hide_local');
$priceFrom = $this->getInput('priceFrom');
$priceTo = $this->getInput('priceFrom');
/* Even if the original website uses POST with the search page, GET works too */
$url = $this->i8n('bridge-uri')
. '/search/advanced?q='
. urlencode($q)
. '&hide_expired='. $hide_expired
. '&hide_local='. $hide_local
. '&priceFrom='. $priceFrom
. '&priceTo='. $priceTo
/* Some default parameters
* search_fields : Search in Titres & Descriptions & Codes
* sort_by : Sort the search by new deals
* time_frame : Search will not be on a limited timeframe
*/
. '&search_fields[]=1&search_fields[]=2&search_fields[]=3&sort_by=new&time_frame=0';
$this->collectDeals($url);
}
/**
* Get the Deal data using the given URL
*/
public function collectDeals($url){
$html = getSimpleHTMLDOM($url)
or returnServerError($this->i8n('request-error'));
$list = $html->find('article[id]');
// Deal Image Link CSS Selector
$selectorImageLink = implode(
' ', /* Notice this is a space! */
array(
'cept-thread-image-link',
'imgFrame',
'imgFrame--noBorder',
'thread-listImgCell',
)
);
// Deal Link CSS Selector
$selectorLink = implode(
' ', /* Notice this is a space! */
array(
'cept-tt',
'thread-link',
'linkPlain',
)
);
// Deal Hotness CSS Selector
$selectorHot = implode(
' ', /* Notice this is a space! */
array(
'flex',
'flex--align-c',
'flex--justify-space-between',
'space--b-2',
)
);
// Deal Description CSS Selector
$selectorDescription = implode(
' ', /* Notice this is a space! */
array(
'cept-description-container',
'overflow--wrap-break',
'size--all-s',
'size--fromW3-m'
)
);
// Deal Date CSS Selector
$selectorDate = implode(
' ', /* Notice this is a space! */
array(
'size--all-s',
'flex',
'flex--justify-e',
'flex--grow-1',
)
);
// If there is no results, we don't parse the content because it display some random deals
$noresult = $html->find('h3[class=size--all-l size--fromW2-xl size--fromW3-xxl]', 0);
if ($noresult != null && strpos($noresult->plaintext, $this->i8n('no-results')) !== false) {
$this->items = array();
} else {
foreach ($list as $deal) {
$item = array();
$item['uri'] = $deal->find('div[class=threadGrid-title]', 0)->find('a', 0)->href;
$item['title'] = $deal->find('a[class*='. $selectorLink .']', 0
)->plaintext;
$item['author'] = $deal->find('span.thread-username', 0)->plaintext;
$item['content'] = '<table><tr><td><a href="'
. $deal->find(
'a[class*='. $selectorImageLink .']', 0)->href
. '"><img src="'
. $this->getImage($deal)
. '"/></td><td><h2><a href="'
. $deal->find('a[class*='. $selectorLink .']', 0)->href
. '">'
. $deal->find('a[class*='. $selectorLink .']', 0)->innertext
. '</a></h2>'
. $this->getPrice($deal)
. $this->getDiscount($deal)
. $this->getShipsFrom($deal)
. $this->getShippingCost($deal)
. $this->GetSource($deal)
. $deal->find('div[class*='. $selectorDescription .']', 0)->innertext
. '</td><td>'
. $deal->find('div[class='. $selectorHot .']', 0)->children(0)->outertext
. '</td></table>';
$dealDateDiv = $deal->find('div[class*='. $selectorDate .']', 0)
->find('span[class=hide--toW3]');
$itemDate = end($dealDateDiv)->plaintext;
// In case of a Local deal, there is no date, but we can use
// this case for other reason (like date not in the last field)
if ($this->contains($itemDate, $this->i8n('localdeal'))) {
$item['timestamp'] = time();
} else if ($this->contains($itemDate, $this->i8n('relative-date-indicator'))) {
$item['timestamp'] = $this->relativeDateToTimestamp($itemDate);
} else {
$item['timestamp'] = $this->parseDate($itemDate);
}
$this->items[] = $item;
}
}
}
/**
* Check if the string $str contains any of the string of the array $arr
* @return boolean true if the string matched anything otherwise false
*/
private function contains($str, array $arr)
{
foreach ($arr as $a) {
if (stripos($str, $a) !== false) {
return true;
}
}
return false;
}
/**
* Get the Price from a Deal if it exists
* @return string String of the deal price
*/
private function getPrice($deal)
{
if ($deal->find(
'span[class*=thread-price]', 0) != null) {
return '<div>'.$this->i8n('price') .' : '
. $deal->find(
'span[class*=thread-price]', 0
)->plaintext
. '</div>';
} else {
return '';
}
}
/**
* Get the Shipping costs from a Deal if it exists
* @return string String of the deal shipping Cost
*/
private function getShippingCost($deal)
{
if ($deal->find('span[class*=cept-shipping-price]', 0) != null) {
if ($deal->find('span[class*=cept-shipping-price]', 0)->children(0) != null) {
return '<div>'. $this->i8n('shipping') .' : '
. $deal->find('span[class*=cept-shipping-price]', 0)->children(0)->innertext
. '</div>';
} else {
return '<div>'. $this->i8n('shipping') .' : '
. $deal->find('span[class*=cept-shipping-price]', 0)->innertext
. '</div>';
}
} else {
return '';
}
}
/**
* Get the source of a Deal if it exists
* @return string String of the deal source
*/
private function GetSource($deal)
{
if ($deal->find('a[class=text--color-greyShade]', 0) != null) {
return '<div>'. $this->i8n('origin') .' : '
. $deal->find('a[class=text--color-greyShade]', 0)->outertext
. '</div>';
} else {
return '';
}
}
/**
* Get the original Price and discout from a Deal if it exists
* @return string String of the deal original price and discount
*/
private function getDiscount($deal)
{
if ($deal->find('span[class*=mute--text text--lineThrough]', 0) != null) {
$discountHtml = $deal->find('span[class=space--ml-1 size--all-l size--fromW3-xl]', 0);
if ($discountHtml != null) {
$discount = $discountHtml->plaintext;
} else {
$discount = '';
}
return '<div>'. $this->i8n('discount') .' : <span style="text-decoration: line-through;">'
. $deal->find(
'span[class*=mute--text text--lineThrough]', 0
)->plaintext
. '</span>&nbsp;'
. $discount
. '</div>';
} else {
return '';
}
}
/**
* Get the Picture URL from a Deal if it exists
* @return string String of the deal Picture URL
*/
private function getImage($deal)
{
$selectorLazy = implode(
' ', /* Notice this is a space! */
array(
'thread-image',
'width--all-auto',
'height--all-auto',
'imgFrame-img',
'cept-thread-img',
'img--dummy',
'js-lazy-img'
)
);
$selectorPlain = implode(
' ', /* Notice this is a space! */
array(
'thread-image',
'width--all-auto',
'height--all-auto',
'imgFrame-img',
'cept-thread-img'
)
);
if ($deal->find('img[class='. $selectorLazy .']', 0) != null) {
return json_decode(
html_entity_decode(
$deal->find('img[class='. $selectorLazy .']', 0)
->getAttribute('data-lazy-img')))->{'src'};
} else {
return $deal->find('img[class*='. $selectorPlain .']', 0 )->src;
}
}
/**
* Get the originating country from a Deal if it exists
* @return string String of the deal originating country
*/
private function getShipsFrom($deal)
{
$selector = implode(
' ', /* Notice this is a space! */
array(
'meta-ribbon',
'overflow--wrap-off',
'space--l-3',
'text--color-greyShade'
)
);
if ($deal->find('span[class='. $selector .']', 0) != null) {
return '<div>'
. $deal->find('span[class='. $selector .']', 0)->children(2)->plaintext
. '</div>';
} else {
return '';
}
}
/**
* Transforms a local date into a timestamp
* @return int timestamp of the input date
*/
private function parseDate($string)
{
$month_local = $this->i8n('local-months');
$month_en = array(
'January',
'February',
'March',
'April',
'May',
'June',
'July',
'August',
'September',
'October',
'November',
'December'
);
// A date can be prfixed with some words, we remove theme
$string = $this->removeDatePrefixes($string);
// We translate the local months name in the english one
$date_str = trim(str_replace($month_local, $month_en, $string));
// If the date does not contain any year, we add the current year
if (!preg_match('/[0-9]{4}/', $string)) {
$date_str .= ' ' . date('Y');
}
// Add the Hour and minutes
$date_str .= ' 00:00';
$date = DateTime::createFromFormat('j F Y H:i', $date_str);
return $date->getTimestamp();
}
/**
* Remove the prefix of a date if it has one
* @return the date without prefiux
*/
private function removeDatePrefixes($string)
{
$string = str_replace($this->i8n('date-prefixes'), array(), $string);
return $string;
}
/**
* Remove the suffix of a relative date if it has one
* @return the relative date without suffixes
*/
private function removeRelativeDateSuffixes($string)
{
if (count($this->i8n('relative-date-ignore-suffix')) > 0) {
$string = preg_replace($this->i8n('relative-date-ignore-suffix'), '', $string);
}
return $string;
}
/**
* Transforms a relative local date into a timestamp
* @return int timestamp of the input date
*/
private function relativeDateToTimestamp($str) {
$date = new DateTime();
// In case of update date, replace it by the regular relative date first word
$str = str_replace($this->i8n('relative-date-alt-prefixes'), $this->i8n('local-time-relative')[0], $str);
$str = $this->removeRelativeDateSuffixes($str);
$search = $this->i8n('local-time-relative');
$replace = array(
'-',
'minute',
'hour',
'day',
'month',
'year',
''
);
$date->modify(str_replace($search, $replace, $str));
return $date->getTimestamp();
}
/**
* Returns the RSS Feed title according to the parameters
* @return string the RSS feed Tiyle
*/
public function getName(){
switch($this->queriedContext) {
case $this->i8n('context-keyword'):
return $this->i8n('bridge-name') . ' - '. $this->i8n('title-keyword') .' : '. $this->getInput('q');
break;
case $this->i8n('context-group'):
$values = $this->getParameters()[$this->i8n('context-group')]['group']['values'];
$group = array_search($this->getInput('group'), $values);
return $this->i8n('bridge-name') . ' - '. $this->i8n('title-group'). ' : '. $group;
break;
default: // Return default value
return static::NAME;
}
}
/**
* This is some "localisation" function that returns the needed content using
* the "$lang" class variable in the local class
* @return various the local content needed
*/
public function i8n($key)
{
if (array_key_exists($key, $this->lang)) {
return $this->lang[$key];
} else {
return null;
}
}
}

View File

@@ -1,49 +1,46 @@
<?php
class DemoBridge extends BridgeAbstract{
class DemoBridge extends BridgeAbstract {
const MAINTAINER = "teromene";
const NAME = "DemoBridge";
const URI = "http://github.com/rss-bridge/rss-bridge";
const DESCRIPTION = "Bridge used for demos";
const MAINTAINER = 'teromene';
const NAME = 'DemoBridge';
const URI = 'http://github.com/rss-bridge/rss-bridge';
const DESCRIPTION = 'Bridge used for demos';
const PARAMETERS = array(
'testCheckbox' => array(
'testCheckbox'=>array(
'type'=>'checkbox',
'name'=>'test des checkbox'
)
),
const PARAMETERS = array(
'testCheckbox' => array(
'testCheckbox' => array(
'type' => 'checkbox',
'name' => 'test des checkbox'
)
),
'testList' => array(
'testList' => array(
'type' => 'list',
'name' => 'test des listes',
'values' => array(
'Test' => 'test',
'Test 2' => 'test2'
)
)
),
'testNumber' => array(
'testNumber' => array(
'type' => 'number',
'name' => 'test des numéros',
'exampleValue' => '1515632'
)
)
);
'testList' => array(
'testList'=>array(
'type'=>'list',
'name'=>'test des listes',
'values'=>array(
'Test'=>'test',
'Test 2'=>'test2'
)
)
),
public function collectData(){
'testNumber' => array(
'testNumber'=>array(
'type'=>'number',
'name'=>'test des numéros',
'exampleValue'=>'1515632'
)
)
);
$item = array();
$item['author'] = 'Me!';
$item['title'] = 'Test';
$item['content'] = 'Awesome content !';
$item['id'] = 'Lalala';
$item['uri'] = 'http://example.com/test';
public function collectData(){
$item = array();
$item['author'] = "Me!";
$item['title'] = "Test";
$item['content'] = "Awesome content !";
$item['id'] = "Lalala";
$item['uri'] = "http://example.com/test";
$this->items[] = $item;
}
$this->items[] = $item;
}
}

166
bridges/DemonoidBridge.php Normal file
View File

@@ -0,0 +1,166 @@
<?php
class DemonoidBridge extends BridgeAbstract {
const MAINTAINER = 'metaMMA';
const NAME = 'Demonoid';
const URI = 'https://www.demonoid.pw/';
const DESCRIPTION = 'Returns results from search';
const PARAMETERS = array(array(
'q' => array(
'name' => 'keywords',
'exampleValue' => 'keyword1 keyword2…',
'required' => true,
),
'category' => array(
'name' => 'Category',
'type' => 'list',
'values' => array(
'All' => 0,
'Movies' => 1,
'Music' => 2,
'TV' => 3,
'Games' => 4,
'Applications' => 5,
'Pictures' => 8,
'Anime' => 9,
'Comics' => 10,
'Books' => 11,
'Audiobooks' => 17
)
)
), array(
'catOnly' => array(
'name' => 'Category',
'type' => 'list',
'values' => array(
'All' => 0,
'Movies' => 1,
'Music' => 2,
'TV' => 3,
'Games' => 4,
'Applications' => 5,
'Pictures' => 8,
'Anime' => 9,
'Comics' => 10,
'Books' => 11,
'Audiobooks' => 17
)
)
), array(
'userid' => array(
'name' => 'user id',
'exampleValue' => '00000',
'required' => true,
'type' => 'number'
),
'category' => array(
'name' => 'Category',
'type' => 'list',
'values' => array(
'All' => 0,
'Movies' => 1,
'Music' => 2,
'TV' => 3,
'Games' => 4,
'Applications' => 5,
'Pictures' => 8,
'Anime' => 9,
'Comics' => 10,
'Books' => 11,
'Audiobooks' => 17
)
)
)
);
public function collectData() {
if(!empty($this->getInput('q'))) {
$html = getSimpleHTMLDOM(
self::URI .
'files/?category=' .
rawurlencode($this->getInput('category')) .
'&subcategory=All&quality=All&seeded=2&external=2&query=' .
urlencode($this->getInput('q')) .
'&uid=0&sort='
) or returnServerError('Could not request Demonoid.');
} elseif(!empty($this->getInput('catOnly'))) {
$html = getSimpleHTMLDOM(
self::URI .
'files/?uid=0&category=' .
rawurlencode($this->getInput('catOnly')) .
'&subcategory=0&language=0&seeded=2&quality=0&query=&sort='
) or returnServerError('Could not request Demonoid.');
} elseif(!empty($this->getInput('userid'))) {
$html = getSimpleHTMLDOM(
self::URI .
'files/?uid=' .
rawurlencode($this->getInput('userid')) .
'&seeded=2'
) or returnServerError('Could not request Demonoid.');
} else {
returnServerError('Invalid parameters !');
}
if(preg_match('~No torrents found~', $html)) {
return;
}
$table = $html->find('td[class=ctable_content_no_pad]', 0);
$cursorCount = 4;
$elementCount = 0;
while($elementCount != 40) {
$elementCount++;
$currentElement = $table->find('tr', $cursorCount);
if(preg_match('~items total~', $currentElement)) {
break;
}
$item = array();
//Do we have a date ?
if(preg_match('~Added.*?(.*)~', $currentElement->plaintext, $dateStr)) {
if(preg_match('~today~', $dateStr[0])) {
date_default_timezone_set('UTC');
$timestamp = mktime(0, 0, 0, gmdate('n'), gmdate('j'), gmdate('Y'));
} else {
preg_match('~(?<=ed on ).*\d+~', $currentElement->plaintext, $fullDateStr);
date_default_timezone_set('UTC');
$dateObj = strptime($fullDateStr[0], '%A, %b %d, %Y');
$timestamp = mktime(0, 0, 0, $dateObj['tm_mon'] + 1, $dateObj['tm_mday'], 1900 + $dateObj['tm_year']);
}
$cursorCount++;
}
$content = $table->find('tr', $cursorCount)->find('a', 1);
$cursorCount++;
$torrentInfo = $table->find('tr', $cursorCount);
$item['timestamp'] = $timestamp;
$item['title'] = $content->plaintext;
$item['id'] = self::URI . $content->href;
$item['uri'] = self::URI . $content->href;
$item['author'] = $torrentInfo->find('a[class=user]', 0)->plaintext;
$item['seeders'] = $torrentInfo->find('font[class=green]', 0)->plaintext;
$item['leechers'] = $torrentInfo->find('font[class=red]', 0)->plaintext;
$item['size'] = $torrentInfo->find('td', 3)->plaintext;
$item['content'] = 'Uploaded by ' . $item['author']
. ' , Size ' . $item['size']
. '<br>seeders: '
. $item['seeders']
. ' | leechers: '
. $item['leechers']
. '<br><a href="'
. $item['id']
. '">info page</a>';
$this->items[] = $item;
$cursorCount++;
}
}
}

View File

@@ -1,11 +1,11 @@
<?php
class DeveloppezDotComBridge extends FeedExpander {
const MAINTAINER = "polopollo";
const NAME = "Developpez.com Actus (FR)";
const URI = "http://www.developpez.com/";
const MAINTAINER = 'polopollo';
const NAME = 'Developpez.com Actus (FR)';
const URI = 'https://www.developpez.com/';
const CACHE_TIMEOUT = 1800; // 30min
const DESCRIPTION = "Returns the 15 newest posts from DeveloppezDotCom (full text).";
const DESCRIPTION = 'Returns the 15 newest posts from DeveloppezDotCom (full text).';
public function collectData(){
$this->collectExpandableDatas(self::URI . 'index/rss', 15);
@@ -13,19 +13,13 @@ class DeveloppezDotComBridge extends FeedExpander {
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
$item['content'] = $this->DeveloppezDotComExtractContent($item['uri']);
$item['content'] = $this->extractContent($item['uri']);
return $item;
}
private function DeveloppezDotComStripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
$string = str_replace(']]>', '', $string);
return $string;
}
// F***ing quotes from Microsoft Word badly encoded, here was the trick:
// http://stackoverflow.com/questions/1262038/how-to-replace-microsoft-encoded-quotes-in-php
private function convert_smart_quotes($string)
private function convertSmartQuotes($string)
{
$search = array(chr(145),
chr(146),
@@ -33,18 +27,20 @@ class DeveloppezDotComBridge extends FeedExpander {
chr(148),
chr(151));
$replace = array("'",
"'",
'"',
'"',
'-');
$replace = array(
"'",
"'",
'"',
'"',
'-'
);
return str_replace($search, $replace, $string);
}
private function DeveloppezDotComExtractContent($url) {
private function extractContent($url){
$articleHTMLContent = getSimpleHTMLDOMCached($url);
$text = $this->convert_smart_quotes($articleHTMLContent->find('div.content', 0)->innertext);
$text = $this->convertSmartQuotes($articleHTMLContent->find('div.content', 0)->innertext);
$text = utf8_encode($text);
return trim($text);
}

120
bridges/DiceBridge.php Normal file
View File

@@ -0,0 +1,120 @@
<?php
class DiceBridge extends BridgeAbstract {
const MAINTAINER = 'rogerdc';
const NAME = 'Dice Unofficial RSS';
const URI = 'https://www.dice.com/';
const DESCRIPTION = 'The Unofficial Dice RSS';
// const CACHE_TIMEOUT = 86400; // 1 day
const PARAMETERS = array(array(
'for_one' => array(
'name' => 'With at least one of the words',
'required' => false,
),
'for_all' => array(
'name' => 'With all of the words',
'required' => false,
),
'for_exact' => array(
'name' => 'With the exact phrase',
'required' => false,
),
'for_none' => array(
'name' => 'With none of these words',
'required' => false,
),
'for_jt' => array(
'name' => 'Within job title',
'required' => false,
),
'for_com' => array(
'name' => 'Within company name',
'required' => false,
),
'for_loc' => array(
'name' => 'City, State, or ZIP code',
'required' => false,
),
'radius' => array(
'name' => 'Radius in miles',
'type' => 'list',
'required' => false,
'values' => array(
'Exact Location' => 'El',
'Within 5 miles' => '5',
'Within 10 miles' => '10',
'Within 20 miles' => '20',
'Within 30 miles' => '0',
'Within 40 miles' => '40',
'Within 50 miles' => '50',
'Within 75 miles' => '75',
'Within 100 miles' => '100',
),
'defaultValue' => '0',
),
'jtype' => array(
'name' => 'Job type',
'type' => 'list',
'required' => false,
'values' => array(
'Full-Time' => 'Full Time',
'Part-Time' => 'Part Time',
'Contract - Independent' => 'Contract Independent',
'Contract - W2' => 'Contract W2',
'Contract to Hire - Independent' => 'C2H Independent',
'Contract to Hire - W2' => 'C2H W2',
'Third Party - Contract - Corp-to-Corp' => 'Contract Corp-To-Corp',
'Third Party - Contract to Hire - Corp-to-Corp' => 'C2H Corp-To-Corp',
),
'defaultValue' => 'Full Time',
),
'telecommute' => array(
'name' => 'Telecommute',
'type' => 'checkbox',
),
));
public function collectData() {
$uri = 'https://www.dice.com/jobs/advancedResult.html';
$uri .= '?for_one=' . urlencode($this->getInput('for_one'));
$uri .= '&for_all=' . urlencode($this->getInput('for_all'));
$uri .= '&for_exact=' . urlencode($this->getInput('for_exact'));
$uri .= '&for_none=' . urlencode($this->getInput('for_none'));
$uri .= '&for_jt=' . urlencode($this->getInput('for_jt'));
$uri .= '&for_com=' . urlencode($this->getInput('for_com'));
$uri .= '&for_loc=' . urlencode($this->getInput('for_loc'));
if ($this->getInput('jtype')) {
$uri .= '&jtype=' . urlencode($this->getInput('jtype'));
}
$uri .= '&sort=date&limit=100';
$uri .= '&radius=' . urlencode($this->getInput('radius'));
if ($this->getInput('telecommute')) {
$uri .= '&telecommute=true';
}
$html = getSimpleHTMLDOM($uri)
or returnServerError('Could not request Dice.');
foreach($html->find('div.complete-serp-result-div') as $element) {
$item = array();
// Title
$masterLink = $element->find('a[id^=position]', 0);
$item['title'] = $masterLink->title;
// URL
$uri = $masterLink->href;
// $uri = substr($uri, 0, strrpos($uri, '?'));
$item['uri'] = substr($uri, 0, strrpos($uri, '?'));
// ID
$item['id'] = $masterLink->value;
// Image
$image = $element->find('img', 0);
if ($image)
$item['image'] = $image->getAttribute('src');
// Content
$shortdesc = $element->find('.shortdesc', '0');
$shortdesc = ($shortdesc) ? $shortdesc->innertext : '';
$item['content'] = $shortdesc;
$this->items[] = $item;
}
}
}

View File

@@ -1,36 +1,36 @@
<?php
class DilbertBridge extends BridgeAbstract {
const MAINTAINER = 'kranack';
const NAME = 'Dilbert Daily Strip';
const URI = 'http://dilbert.com';
const CACHE_TIMEOUT = 21600; // 6h
const DESCRIPTION = 'The Unofficial Dilbert Daily Comic Strip';
const MAINTAINER = 'kranack';
const NAME = 'Dilbert Daily Strip';
const URI = 'http://dilbert.com';
const CACHE_TIMEOUT = 21600; // 6h
const DESCRIPTION = 'The Unofficial Dilbert Daily Comic Strip';
public function collectData(){
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI()) or returnServerError('Could not request Dilbert: '.$this->getURI());
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request Dilbert: ' . $this->getURI());
foreach ($html->find('section.comic-item') as $element) {
foreach($html->find('section.comic-item') as $element) {
$img = $element->find('img', 0);
$link = $element->find('a', 0);
$comic = $img->src;
$title = $link->alt;
$url = $link->href;
$date = substr($url, 25);
if (empty($title))
$title = 'Dilbert Comic Strip on '.$date;
$date = strtotime($date);
$img = $element->find('img', 0);
$link = $element->find('a', 0);
$comic = $img->src;
$title = $link->alt;
$url = $link->href;
$date = substr($url, 25);
if (empty($title))
$title = 'Dilbert Comic Strip on ' . $date;
$date = strtotime($date);
$item = array();
$item['uri'] = $url;
$item['title'] = $title;
$item['author'] = 'Scott Adams';
$item['timestamp'] = $date;
$item['content'] = '<img src="'.$comic.'" alt="'.$img->alt.'" />';
$this->items[] = $item;
}
}
$item = array();
$item['uri'] = $url;
$item['title'] = $title;
$item['author'] = 'Scott Adams';
$item['timestamp'] = $date;
$item['content'] = '<img src="' . $comic . '" alt="' . $img->alt . '" />';
$this->items[] = $item;
}
}
}
?>

112
bridges/DiscogsBridge.php Normal file
View File

@@ -0,0 +1,112 @@
<?php
class DiscogsBridge extends BridgeAbstract {
const MAINTAINER = 'teromene';
const NAME = 'DiscogsBridge';
const URI = 'https://www.discogs.com/';
const DESCRIPTION = 'Returns releases from discogs';
const PARAMETERS = array(
'Artist Releases' => array(
'artistid' => array(
'name' => 'Artist ID',
'type' => 'number',
)
),
'Label Releases' => array(
'labelid' => array(
'name' => 'Label ID',
'type' => 'number',
)
),
'User Wantlist' => array(
'username_wantlist' => array(
'name' => 'Username',
'type' => 'text',
)
),
'User Folder' => array(
'username_folder' => array(
'name' => 'Username',
'type' => 'text',
),
'folderid' => array(
'name' => 'Folder ID',
'type' => 'number',
)
)
);
public function collectData() {
if(!empty($this->getInput('artistid')) || !empty($this->getInput('labelid'))) {
if(!empty($this->getInput('artistid'))) {
$data = getContents('https://api.discogs.com/artists/'
. $this->getInput('artistid')
. '/releases?sort=year&sort_order=desc')
or returnServerError('Unable to query discogs !');
} elseif(!empty($this->getInput('labelid'))) {
$data = getContents('https://api.discogs.com/labels/'
. $this->getInput('labelid')
. '/releases?sort=year&sort_order=desc')
or returnServerError('Unable to query discogs !');
}
$jsonData = json_decode($data, true);
foreach($jsonData['releases'] as $release) {
$item = array();
$item['author'] = $release['artist'];
$item['title'] = $release['title'];
$item['id'] = $release['id'];
$resId = array_key_exists('main_release', $release) ? $release['main_release'] : $release['id'];
$item['uri'] = self::URI . $this->getInput('artistid') . '/release/' . $resId;
$item['timestamp'] = DateTime::createFromFormat('Y', $release['year'])->getTimestamp();
$item['content'] = $item['author'] . ' - ' . $item['title'];
$this->items[] = $item;
}
} elseif(!empty($this->getInput('username_wantlist')) || !empty($this->getInput('username_folder'))) {
if(!empty($this->getInput('username_wantlist'))) {
$data = getContents('https://api.discogs.com/users/'
. $this->getInput('username_wantlist')
. '/wants?sort=added&sort_order=desc')
or returnServerError('Unable to query discogs !');
$jsonData = json_decode($data, true)['wants'];
} elseif(!empty($this->getInput('username_folder'))) {
$data = getContents('https://api.discogs.com/users/'
. $this->getInput('username_folder')
. '/collection/folders/'
. $this->getInput('folderid')
.'/releases?sort=added&sort_order=desc')
or returnServerError('Unable to query discogs !');
$jsonData = json_decode($data, true)['releases'];
}
foreach($jsonData as $element) {
$infos = $element['basic_information'];
$item = array();
$item['title'] = $infos['title'];
$item['author'] = $infos['artists'][0]['name'];
$item['id'] = $infos['artists'][0]['id'];
$item['uri'] = self::URI . $infos['artists'][0]['id'] . '/release/' . $infos['id'];
$item['timestamp'] = strtotime($element['date_added']);
$item['content'] = $item['author'] . ' - ' . $item['title'];
$this->items[] = $item;
}
}
}
public function getURI() {
return self::URI;
}
public function getName() {
return static::NAME;
}
}

View File

@@ -1,11 +1,9 @@
<?php
require_once('Shimmie2Bridge.php');
class DollbooruBridge extends Shimmie2Bridge{
const MAINTAINER = "mitsukarenai";
const NAME = "Dollbooru";
const URI = "http://dollbooru.org/";
const DESCRIPTION = "Returns images from given page";
class DollbooruBridge extends Shimmie2Bridge {
const MAINTAINER = 'mitsukarenai';
const NAME = 'Dollbooru';
const URI = 'http://dollbooru.org/';
const DESCRIPTION = 'Returns images from given page';
}

View File

@@ -0,0 +1,91 @@
<?php
class DribbbleBridge extends BridgeAbstract {
const MAINTAINER = 'quentinus95';
const NAME = 'Dribbble popular shots';
const URI = 'https://dribbble.com';
const CACHE_TIMEOUT = 1800;
const DESCRIPTION = 'Returns the newest popular shots from Dribbble.';
public function collectData(){
$html = getSimpleHTMLDOM(self::URI . '/shots')
or returnServerError('Error while downloading the website content');
$json = $this->loadEmbeddedJsonData($html);
foreach($html->find('li[id^="screenshot-"]') as $shot) {
$item = [];
$additional_data = $this->findJsonForShot($shot, $json);
if ($additional_data === null) {
$item['uri'] = self::URI . $shot->find('a', 0)->href;
$item['title'] = $shot->find('.dribbble-over strong', 0)->plaintext;
} else {
$item['timestamp'] = strtotime($additional_data['published_at']);
$item['uri'] = self::URI . $additional_data['path'];
$item['title'] = $additional_data['title'];
}
$item['author'] = trim($shot->find('.attribution-user a', 0)->plaintext);
$description = $shot->find('.comment', 0);
$item['content'] = $description === null ? '' : $description->plaintext;
$preview_path = $shot->find('picture source', 0)->attr['srcset'];
$item['content'] .= $this->getImageTag($preview_path, $item['title']);
$item['enclosures'] = [$this->getFullSizeImagePath($preview_path)];
$this->items[] = $item;
}
}
private function loadEmbeddedJsonData($html){
$json = [];
$scripts = $html->find('script');
foreach($scripts as $script) {
if(strpos($script->innertext, 'newestShots') !== false) {
// fix single quotes
$script->innertext = str_replace('\'', '"', $script->innertext);
// fix JavaScript JSON (why do they not adhere to the standard?)
$script->innertext = preg_replace('/(\w+):/i', '"\1":', $script->innertext);
// find beginning of JSON array
$start = strpos($script->innertext, '[');
// find end of JSON array, compensate for missing character!
$end = strpos($script->innertext, '];') + 1;
// convert JSON to PHP array
$json = json_decode(substr($script->innertext, $start, $end - $start), true);
break;
}
}
return $json;
}
private function findJsonForShot($shot, $json){
foreach($json as $element) {
if(strpos($shot->getAttribute('id'), (string)$element['id']) !== false) {
return $element;
}
}
return null;
}
private function getImageTag($preview_path, $title){
return sprintf(
'<br /> <a href="%s"><img src="%s" alt="%s" /></a>',
$this->getFullSizeImagePath($preview_path),
$preview_path,
$title
);
}
private function getFullSizeImagePath($preview_path){
return str_replace('_1x', '', $preview_path);
}
}

View File

@@ -1,41 +1,42 @@
<?php
class DuckDuckGoBridge extends BridgeAbstract{
class DuckDuckGoBridge extends BridgeAbstract {
const MAINTAINER = "Astalaseven";
const NAME = "DuckDuckGo";
const URI = "https://duckduckgo.com/";
const MAINTAINER = 'Astalaseven';
const NAME = 'DuckDuckGo';
const URI = 'https://duckduckgo.com/';
const CACHE_TIMEOUT = 21600; // 6h
const DESCRIPTION = "Returns results from DuckDuckGo.";
const DESCRIPTION = 'Returns results from DuckDuckGo.';
const SORT_DATE = '+sort:date';
const SORT_RELEVANCE = '';
const PARAMETERS = array( array(
'u'=>array(
'name'=>'keyword',
'required'=>true),
'sort'=>array(
'name'=>'sort by',
'type'=>'list',
'required'=>false,
'values'=>array(
'date'=>self::SORT_DATE,
'relevance'=>self::SORT_RELEVANCE
),
'defaultValue'=>self::SORT_DATE
)
));
const PARAMETERS = array( array(
'u' => array(
'name' => 'keyword',
'required' => true
),
'sort' => array(
'name' => 'sort by',
'type' => 'list',
'required' => false,
'values' => array(
'date' => self::SORT_DATE,
'relevance' => self::SORT_RELEVANCE
),
'defaultValue' => self::SORT_DATE
)
));
public function collectData(){
$html = getSimpleHTMLDOM(self::URI.'html/?q='.$this->getInput('u').$this->getInput('sort'))
or returnServerError('Could not request DuckDuckGo.');
public function collectData(){
$html = getSimpleHTMLDOM(self::URI . 'html/?kd=-1&q=' . $this->getInput('u') . $this->getInput('sort'))
or returnServerError('Could not request DuckDuckGo.');
foreach($html->find('div.results_links') as $element) {
$item = array();
$item['uri'] = $element->find('a', 0)->href;
$item['title'] = $element->find('a', 1)->innertext;
$item['content'] = $element->find('div.snippet', 0)->plaintext;
$this->items[] = $item;
}
}
foreach($html->find('div.results_links') as $element) {
$item = array();
$item['uri'] = $element->find('a', 0)->href;
$item['title'] = $element->find('a', 1)->innertext;
$item['content'] = $element->find('div.snippet', 0)->plaintext;
$this->items[] = $item;
}
}
}

142
bridges/ETTVBridge.php Normal file
View File

@@ -0,0 +1,142 @@
<?php
class ETTVBridge extends BridgeAbstract {
const MAINTAINER = 'GregThib';
const NAME = 'ETTV';
const URI = 'https://www.ettv.tv/';
const DESCRIPTION = 'Returns list of 20 latest torrents for a specific search.';
const CACHE_TIMEOUT = 14400; // 4 hours
const PARAMETERS = array( array(
'query' => array(
'name' => 'Keywords',
'required' => true
),
'cat' => array(
'type' => 'list',
'name' => 'Category',
'values' => array(
'(ALL TYPES)' => '0',
'Anime: Movies' => '73',
'Anime: Dubbed/Subbed' => '74',
'Anime: Others' => '75',
'Books: Ebooks' => '53',
'Books: Magazines' => '54',
'Books: Comics' => '55',
'Books: Audio' => '56',
'Books: Others' => '68',
'Games: Windows' => '57',
'Games: Android' => '58',
'Games: Others' => '71',
'Movies: HD 1080p' => '1',
'Movies: HD 720p' => '2',
'Movies: UltraHD/4K' => '3',
'Movies: XviD' => '42',
'Movies: X264/H264' => '47',
'Movies: 3D' => '49',
'Movies: Dubs/Dual Audio' => '51',
'Movies: CAM/TS' => '65',
'Movies: BluRay Disc/Remux' => '66',
'Movies: DVDR' => '67',
'Movies: HEVC/x265' => '76',
'Music: MP3' => '59',
'Music: FLAC' => '60',
'Music: Music Videos' => '61',
'Music: Others' => '69',
'Software: Windows' => '62',
'Software: Android' => '63',
'Software: Mac' => '64',
'Software: Others' => '70',
'TV: HD/X264/H264' => '41',
'TV: SD/X264/H264' => '5',
'TV: TV Packs' => '7',
'TV: SD/XVID' => '50',
'TV: Sport' => '72',
'TV: HEVC/x265' => '77',
'Unsorted: Unsorted' => '78'
),
'defaultValue' => '(ALL TYPES)'
),
'status' => array(
'type' => 'list',
'name' => 'Status',
'values' => array(
'Active Transfers' => '0',
'Included Dead' => '1',
'Only Dead' => '2'
),
'defaultValue' => 'Included Dead'
),
'lang' => array(
'type' => 'list',
'name' => 'Lang',
'values' => array(
'(ALL)' => '0',
'Arabic' => '17',
'Chinese ' => '10',
'Danish' => '13',
'Dutch' => '11',
'English' => '1',
'Finnish' => '18',
'French' => '2',
'German' => '3',
'Greek' => '15',
'Hindi' => '8',
'Italian' => '4',
'Japanese' => '5',
'Korean' => '9',
'Polish' => '14',
'Russian' => '7',
'Spanish' => '6',
'Turkish' => '16'
),
'defaultValue' => '(ALL)'
)
));
public function collectData(){
// No control on inputs, because all have defaultValue set
$query_str = 'torrents-search.php';
$query_str .= '?search=' . urlencode('+'.str_replace(' ', ' +', $this->getInput('query')));
$query_str .= '&cat=' . $this->getInput('cat');
$query_str .= 'incldead&=' . $this->getInput('status');
$query_str .= '&lang=' . $this->getInput('lang');
$query_str .= '&sort=id&order=desc';
// Get results page
$html = getSimpleHTMLDOM(self::URI . $query_str)
or returnServerError('Could not request ' . $this->getName());
// Loop on each entry
foreach($html->find('table.table tr') as $element) {
if($element->parent->tag == 'thead') continue;
$entry = $element->find('td', 1)->find('a', 0);
// retrieve result page to get more details
$link = rtrim(self::URI, '/') . $entry->href;
$page = getSimpleHTMLDOM($link)
or returnServerError('Could not request page ' . $link);
// get details & download links
$details = $page->find('fieldset.download table', 0); // WHAT?? It should be the second one…
$dllinks = $page->find('div#downloadbox table', 0);
// fill item
$item = array();
$item['author'] = $details->children(6)->children(1)->plaintext;
$item['title'] = $entry->title;
$item['uri'] = $dllinks->children(0)->children(0)->children(0)->href;
$item['timestamp'] = strtotime($details->children(7)->children(1)->plaintext);
$item['content'] = '';
$item['content'] .= '<br/><b>Name: </b>' . $details->children(0)->children(1)->innertext;
$item['content'] .= '<br/><b>Lang: </b>' . $details->children(3)->children(1)->innertext;
$item['content'] .= '<br/><b>Size: </b>' . $details->children(4)->children(1)->innertext;
$item['content'] .= '<br/><b>Hash: </b>' . $details->children(5)->children(1)->innertext;
foreach($dllinks->children(0)->children(1)->find('a') as $dl) {
$item['content'] .= '<br/>' . $dl->outertext;
}
$item['content'] .= '<br/><br/>' . $details->children(1)->children(0)->innertext;
$this->items[] = $item;
}
}
}

View File

@@ -1,66 +1,67 @@
<?php
class EZTVBridge extends BridgeAbstract{
class EZTVBridge extends BridgeAbstract {
const MAINTAINER = "alexAubin";
const NAME = "EZTV";
const URI = "https://eztv.ch/";
const DESCRIPTION = "Returns list of *recent* torrents for a specific show on EZTV. Get showID from URLs in https://eztv.ch/shows/showID/show-full-name.";
const MAINTAINER = 'alexAubin';
const NAME = 'EZTV';
const URI = 'https://eztv.ch/';
const DESCRIPTION = 'Returns list of *recent* torrents for a specific show
on EZTV. Get showID from URLs in https://eztv.ch/shows/showID/show-full-name.';
const PARAMETERS = array( array(
'i'=>array(
'name'=>'Show ids',
'exampleValue'=>'showID1,showID2,…',
'required'=>true
)
));
const PARAMETERS = array( array(
'i' => array(
'name' => 'Show ids',
'exampleValue' => 'showID1,showID2,…',
'required' => true
)
));
public function collectData(){
// Make timestamp from relative released time in table
function makeTimestamp($relativeReleaseTime){
// Make timestamp from relative released time in table
function makeTimestamp($relativeReleaseTime){
$relativeDays = 0;
$relativeHours = 0;
$relativeDays = 0;
$relativeHours = 0;
foreach (explode(" ",$relativeReleaseTime) as $relativeTimeElement) {
if (substr($relativeTimeElement,-1) == "d") $relativeDays = substr($relativeTimeElement,0,-1);
if (substr($relativeTimeElement,-1) == "h") $relativeHours = substr($relativeTimeElement,0,-1);
}
return mktime(date('h')-$relativeHours,0,0,date('m'),date('d')-$relativeDays,date('Y'));
}
foreach(explode(' ', $relativeReleaseTime) as $relativeTimeElement) {
if(substr($relativeTimeElement, -1) == 'd') $relativeDays = substr($relativeTimeElement, 0, -1);
if(substr($relativeTimeElement, -1) == 'h') $relativeHours = substr($relativeTimeElement, 0, -1);
}
return mktime(date('h') - $relativeHours, 0, 0, date('m'), date('d') - $relativeDays, date('Y'));
}
// Loop on show ids
$showList = explode(",",$this->getInput('i'));
foreach($showList as $showID){
// Loop on show ids
$showList = explode(',', $this->getInput('i'));
foreach($showList as $showID) {
// Get show page
$html = getSimpleHTMLDOM(self::URI.'shows/'.rawurlencode($showID).'/')
or returnServerError('Could not request EZTV for id "'.$showID.'"');
// Get show page
$html = getSimpleHTMLDOM(self::URI . 'shows/' . rawurlencode($showID) . '/')
or returnServerError('Could not request EZTV for id "' . $showID . '"');
// Loop on each element that look like an episode entry...
foreach($html->find('.forum_header_border') as $element) {
// Loop on each element that look like an episode entry...
foreach($html->find('.forum_header_border') as $element) {
// Filter entries that are not episode entries
$ep = $element->find('td',1);
if (empty($ep)) continue;
$epinfo = $ep->find('.epinfo',0);
$released = $element->find('td',3);
if (empty($epinfo)) continue;
if (empty($released->plaintext)) continue;
// Filter entries that are not episode entries
$ep = $element->find('td', 1);
if(empty($ep)) continue;
$epinfo = $ep->find('.epinfo', 0);
$released = $element->find('td', 3);
if(empty($epinfo)) continue;
if(empty($released->plaintext)) continue;
// Filter entries that are older than 1 week
if ($released->plaintext == '&gt;1 week') continue;
// Filter entries that are older than 1 week
if($released->plaintext == '&gt;1 week') continue;
// Fill item
$item = array();
$item['uri'] = self::URI.$epinfo->href;
$item['id'] = $item['uri'];
$item['timestamp'] = makeTimestamp($released->plaintext);
$item['title'] = $epinfo->plaintext;
$item['content'] = $epinfo->alt;
if(isset($item['title']))
$this->items[] = $item;
}
}
// Fill item
$item = array();
$item['uri'] = self::URI . $epinfo->href;
$item['id'] = $item['uri'];
$item['timestamp'] = makeTimestamp($released->plaintext);
$item['title'] = $epinfo->plaintext;
$item['content'] = $epinfo->alt;
if(isset($item['title']))
$this->items[] = $item;
}
}
}
}

View File

@@ -1,21 +1,21 @@
<?php
class EliteDangerousGalnetBridge extends BridgeAbstract
{
const MAINTAINER = "corenting";
const NAME = "Elite: Dangerous Galnet";
const URI = "https://community.elitedangerous.com/galnet/";
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = "Returns the latest page of news from Galnet";
class EliteDangerousGalnetBridge extends BridgeAbstract {
const MAINTAINER = 'corenting';
const NAME = 'Elite: Dangerous Galnet';
const URI = 'https://community.elitedangerous.com/galnet/';
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = 'Returns the latest page of news from Galnet';
public function collectData(){
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Error while downloading the website content');
public function collectData()
{
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Error while downloading the website content');
foreach($html->find('div.article') as $element) {
$item = array();
$uri = $element->find('h3 a', 0)->href;
$uri = self::URI . substr($uri,strlen('/galnet/'));
$uri = self::URI . substr($uri, strlen('/galnet/'));
$item['uri'] = $uri;
$title = $element->find('h3 a', 0)->plaintext;

146
bridges/ElloBridge.php Normal file
View File

@@ -0,0 +1,146 @@
<?php
class ElloBridge extends BridgeAbstract {
const MAINTAINER = 'teromene';
const NAME = 'Ello Bridge';
const URI = 'https://ello.co/';
const CACHE_TIMEOUT = 4800; //2hours
const DESCRIPTION = 'Returns the newest posts for Ello';
const PARAMETERS = array(
'By User' => array(
'u' => array(
'name' => 'Username',
'required' => true,
'title' => 'Username'
)
),
'Search' => array(
's' => array(
'name' => 'Search',
'required' => true,
'title' => 'Search'
)
)
);
public function collectData() {
$header = array(
'Authorization: Bearer ' . $this->getAPIKey()
);
if(!empty($this->getInput('u'))) {
$postData = getContents(self::URI . 'api/v2/users/~' . urlencode($this->getInput('u')) . '/posts', $header) or
returnServerError('Unable to query Ello API.');
} else {
$postData = getContents(self::URI . 'api/v2/posts?terms=' . urlencode($this->getInput('s')), $header) or
returnServerError('Unable to query Ello API.');
}
$postData = json_decode($postData);
$count = 0;
foreach($postData->posts as $post) {
$item = array();
$item['author'] = $this->getUsername($post, $postData);
$item['timestamp'] = strtotime($post->created_at);
$item['title'] = $this->findText($post->summary);
$item['content'] = $this->getPostContent($post->body);
$item['enclosures'] = $this->getEnclosures($post, $postData);
$content = $post->body;
$this->items[] = $item;
$count += 1;
}
}
public function findText($path) {
foreach($path as $summaryElement) {
if($summaryElement->kind == 'text') {
return $summaryElement->data;
}
}
return '';
}
public function getPostContent($path) {
$content = '';
foreach($path as $summaryElement) {
if($summaryElement->kind == 'text') {
$content .= $summaryElement->data;
} elseif ($summaryElement->kind == 'image') {
$alt = '';
if(property_exists($summaryElement->data, 'alt')) {
$alt = $summaryElement->data->alt;
}
$content .= '<img src="' . $summaryElement->data->url . '" alt="' . $alt . '" />';
}
}
return $content;
}
public function getEnclosures($post, $postData) {
$assets = [];
foreach($post->links->assets as $asset) {
foreach($postData->linked->assets as $assetLink) {
if($asset == $assetLink->id) {
$assets[] = $assetLink->attachment->original->url;
break;
}
}
}
return $assets;
}
public function getUsername($post, $postData) {
foreach($postData->linked->users as $user) {
if($user->id == $post->links->author->id) {
return $user->username;
}
}
}
public function getAPIKey() {
$cache = Cache::create('FileCache');
$cache->setPath(CACHE_DIR);
$cache->setParameters(['key']);
$key = $cache->loadData();
if($key == null) {
$keyInfo = getContents(self::URI . 'api/webapp-token') or
returnServerError('Unable to get token.');
$key = json_decode($keyInfo)->token->access_token;
$cache->saveData($key);
}
return $key;
}
public function getName(){
if(!is_null($this->getInput('u'))) {
return $this->getInput('u') . ' - Ello Bridge';
}
return parent::getName();
}
}

View File

@@ -1,22 +1,23 @@
<?php
class ElsevierBridge extends BridgeAbstract{
class ElsevierBridge extends BridgeAbstract {
const MAINTAINER = 'Pierre Mazière';
const NAME = 'Elsevier journals recent articles';
const URI = 'http://www.journals.elsevier.com/';
const CACHE_TIMEOUT = 43200; //12h
const DESCRIPTION = 'Returns the recent articles published in Elsevier journals';
const PARAMETERS = array( array(
'j'=>array(
'name'=>'Journal name',
'required'=>true,
'exampleValue'=>'academic-pediactrics',
'title'=>'Insert html-part of your journal'
)
));
const PARAMETERS = array( array(
'j' => array(
'name' => 'Journal name',
'required' => true,
'exampleValue' => 'academic-pediactrics',
'title' => 'Insert html-part of your journal'
)
));
// Extracts the list of names from an article as string
private function ExtractArticleName ($article){
private function extractArticleName($article){
$names = $article->find('small', 0);
if($names)
return trim($names->plaintext);
@@ -24,9 +25,9 @@ class ElsevierBridge extends BridgeAbstract{
}
// Extracts the timestamp from an article
private function ExtractArticleTimestamp ($article){
private function extractArticleTimestamp($article){
$time = $article->find('.article-info', 0);
if($time){
if($time) {
$timestring = trim($time->plaintext);
/*
The format depends on the age of an article:
@@ -34,11 +35,11 @@ class ElsevierBridge extends BridgeAbstract{
- July 2016
- MayJune 2016
*/
if(preg_match('/\S*(\d+\s\S+\s\d{4})/ims', $timestring, $matches)){
if(preg_match('/\S*(\d+\s\S+\s\d{4})/ims', $timestring, $matches)) {
return strtotime($matches[0]);
} elseif (preg_match('/[A-Za-z]+\-([A-Za-z]+\s\d{4})/ims', $timestring, $matches)){
} elseif (preg_match('/[A-Za-z]+\-([A-Za-z]+\s\d{4})/ims', $timestring, $matches)) {
return strtotime($matches[0]);
} elseif (preg_match('/([A-Za-z]+\s\d{4})/ims', $timestring, $matches)){
} elseif (preg_match('/([A-Za-z]+\s\d{4})/ims', $timestring, $matches)) {
return strtotime($matches[0]);
} else {
return 0;
@@ -48,9 +49,9 @@ class ElsevierBridge extends BridgeAbstract{
}
// Extracts the content from an article
private function ExtractArticleContent ($article){
private function extractArticleContent($article){
$content = $article->find('.article-content', 0);
if($content){
if($content) {
return trim($content->plaintext);
}
return '';
@@ -58,17 +59,17 @@ class ElsevierBridge extends BridgeAbstract{
public function collectData(){
$uri = self::URI . $this->getInput('j') . '/recent-articles/';
$html = getSimpleHTMLDOM($uri) or returnServerError('No results for Elsevier journal '.$this->getInput('j'));
$html = getSimpleHTMLDOM($uri)
or returnServerError('No results for Elsevier journal ' . $this->getInput('j'));
foreach($html->find('.pod-listing') as $article){
foreach($html->find('.pod-listing') as $article) {
$item = array();
$item['uri'] = $article->find('.pod-listing-header>a',0)->getAttribute('href').'?np=y';
$item['title'] = $article->find('.pod-listing-header>a',0)->plaintext;
$item['author'] = $this->ExtractArticleName($article);
$item['timestamp'] = $this->ExtractArticleTimestamp($article);
$item['content'] = $this->ExtractArticleContent($article);
$item['uri'] = $article->find('.pod-listing-header>a', 0)->getAttribute('href') . '?np=y';
$item['title'] = $article->find('.pod-listing-header>a', 0)->plaintext;
$item['author'] = $this->extractArticleName($article);
$item['timestamp'] = $this->extractArticleTimestamp($article);
$item['content'] = $this->extractArticleContent($article);
$this->items[] = $item;
}
}
}
?>

View File

@@ -1,30 +1,37 @@
<?php
class EstCeQuonMetEnProdBridge extends BridgeAbstract {
const MAINTAINER = 'ORelio';
const NAME = 'Est-ce qu\'on met en prod aujourd\'hui ?';
const URI = 'https://www.estcequonmetenprodaujourdhui.info/';
const CACHE_TIMEOUT = 21600; // 6h
const DESCRIPTION = 'Should we put a website in production today? (French)';
const MAINTAINER = 'ORelio';
const NAME = 'Est-ce qu\'on met en prod aujourd\'hui ?';
const URI = 'https://www.estcequonmetenprodaujourdhui.info/';
const CACHE_TIMEOUT = 21600; // 6h
const DESCRIPTION = 'Should we put a website in production today? (French)';
public function collectData(){
function ExtractFromDelimiters($string, $start, $end) {
if (strpos($string, $start) !== false) {
$section_retrieved = substr($string, strpos($string, $start) + strlen($start));
$section_retrieved = substr($section_retrieved, 0, strpos($section_retrieved, $end));
return $section_retrieved;
} return false;
}
public function collectData(){
function extractFromDelimiters($string, $start, $end){
if(strpos($string, $start) !== false) {
$section_retrieved = substr($string, strpos($string, $start) + strlen($start));
$section_retrieved = substr($section_retrieved, 0, strpos($section_retrieved, $end));
return $section_retrieved;
}
$html = getSimpleHTMLDOM($this->getURI()) or returnServerError('Could not request EstCeQuonMetEnProd: '.$this->getURI());
return false;
}
$item = array();
$item['uri'] = $this->getURI().'#'.date('Y-m-d');
$item['title'] = $this->getName();
$item['author'] = 'Nicolas Hoffmann';
$item['timestamp'] = strtotime('today midnight');
$item['content'] = str_replace('src="/', 'src="'.$this->getURI(), trim(ExtractFromDelimiters($html->outertext, '<body role="document">', '<br /><br />')));
$this->items[] = $item;
}
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request EstCeQuonMetEnProd: ' . $this->getURI());
$item = array();
$item['uri'] = $this->getURI() . '#' . date('Y-m-d');
$item['title'] = $this->getName();
$item['author'] = 'Nicolas Hoffmann';
$item['timestamp'] = strtotime('today midnight');
$item['content'] = str_replace(
'src="/',
'src="' . $this->getURI(),
trim(extractFromDelimiters($html->outertext, '<body role="document">', '<br /><br />'))
);
$this->items[] = $item;
}
}
?>

83
bridges/EtsyBridge.php Normal file
View File

@@ -0,0 +1,83 @@
<?php
class EtsyBridge extends BridgeAbstract {
const NAME = 'Etsy search';
const URI = 'https://www.etsy.com';
const DESCRIPTION = 'Returns feeds for search results';
const MAINTAINER = 'logmanoriginal';
const PARAMETERS = array(
array(
'query' => array(
'name' => 'Search query',
'type' => 'text',
'required' => true,
'title' => 'Insert your search term here',
'exampleValue' => 'Enter your search term'
),
'queryextension' => array(
'name' => 'Query extension',
'type' => 'text',
'requied' => false,
'title' => 'Insert additional query parts here
(anything after ?search=<your search query>)',
'exampleValue' => '&explicit=1&locationQuery=2921044'
),
'showimage' => array(
'name' => 'Show image in content',
'type' => 'checkbox',
'requrired' => false,
'title' => 'Activate to show the image in the content',
'defaultValue' => false
)
)
);
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Failed to receive ' . $this->getURI());
$results = $html->find('div.block-grid-item');
foreach($results as $result) {
// Skip banner cards (ads for categories)
if($result->find('a.banner-card'))
continue;
$item = array();
$item['title'] = $result->find('a', 0)->title;
$item['uri'] = $result->find('a', 0)->href;
$item['author'] = $result->find('div.card-shop-name', 0)->plaintext;
$item['content'] = '<p>'
. $result->find('div.card-price', 0)->plaintext
. '</p><p>'
. $result->find('div.card-title', 0)->plaintext
. '</p>';
$image = $result->find('img.placeholder', 0)->src;
if($this->getInput('showimage')) {
$item['content'] .= '<img src="' . $image . '">';
}
$item['enclosures'] = array($image);
$this->items[] = $item;
}
}
public function getURI(){
if(!is_null($this->getInput('query'))) {
$uri = self::URI . '/search?q=' . urlencode($this->getInput('query'));
if(!is_null($this->getInput('queryextension'))) {
$uri .= $this->getInput('queryextension');
}
return $uri;
}
return parent::getURI();
}
}

281
bridges/FB2Bridge.php Normal file
View File

@@ -0,0 +1,281 @@
<?php
class FB2Bridge extends BridgeAbstract {
const MAINTAINER = 'teromene';
const NAME = 'Facebook Alternate';
const URI = 'https://www.facebook.com/';
const CACHE_TIMEOUT = 1000;
const DESCRIPTION = 'Input a page title or a profile log. For a profile log,
please insert the parameter as follow : myExamplePage/132621766841117';
const PARAMETERS = array( array(
'u' => array(
'name' => 'Username',
'required' => true
)
));
public function collectData(){
function extractFromDelimiters($string, $start, $end){
if(strpos($string, $start) !== false) {
$section_retrieved = substr($string, strpos($string, $start) + strlen($start));
$section_retrieved = substr($section_retrieved, 0, strpos($section_retrieved, $end));
return $section_retrieved;
}
return false;
}
//Utility function for cleaning a Facebook link
$unescape_fb_link = function($matches){
if(is_array($matches) && count($matches) > 1) {
$link = $matches[1];
if(strpos($link, '/') === 0)
$link = self::URI . $link . '"';
if(strpos($link, 'facebook.com/l.php?u=') !== false)
$link = urldecode(extractFromDelimiters($link, 'facebook.com/l.php?u=', '&'));
return ' href="' . $link . '"';
}
};
//Utility function for converting facebook emoticons
$unescape_fb_emote = function($matches){
static $facebook_emoticons = array(
'smile' => ':)',
'frown' => ':(',
'tongue' => ':P',
'grin' => ':D',
'gasp' => ':O',
'wink' => ';)',
'pacman' => ':<',
'grumpy' => '>_<',
'unsure' => ':/',
'cry' => ':\'(',
'kiki' => '^_^',
'glasses' => '8-)',
'sunglasses' => 'B-)',
'heart' => '<3',
'devil' => ']:D',
'angel' => '0:)',
'squint' => '-_-',
'confused' => 'o_O',
'upset' => 'xD',
'colonthree' => ':3',
'like' => '&#x1F44D;');
$len = count($matches);
if ($len > 1)
for ($i = 1; $i < $len; $i++)
foreach ($facebook_emoticons as $name => $emote)
if ($matches[$i] === $name)
return $emote;
return $matches[0];
};
if($this->getInput('u') !== null) {
$page = 'https://touch.facebook.com/' . $this->getInput('u');
$cookies = $this->getCookies($page);
$pageID = $this->getPageID($page, $cookies);
if($pageID === null) {
echo <<<EOD
Unable to get the page id. You should consider getting the ID by hand, then importing it into FB2Bridge
EOD;
die();
} elseif($pageID == -1) {
echo <<<EOD
This page is not accessible without being logged in.
EOD;
die();
}
}
//Build the string for the first request
$requestString = 'https://touch.facebook.com/pages_reaction_units/more/?page_id='
. $pageID
. '&cursor={"card_id"%3A"videos"%2C"has_next_page"%3Atrue}&surface=mobile_page_home&unit_count=8';
$fileContent = getContents($requestString);
$articleIndex = 0;
$maxArticle = 3;
$html = $this->buildContent($fileContent);
$author = $this->getInput('u');
foreach($html->find('article') as $content) {
$item = array();
$item['uri'] = 'http://touch.facebook.com'
. $content->find("div[class='_52jc _5qc4 _24u0 _36xo']", 0)->find('a', 0)->getAttribute('href');
if($content->find('header', 0) !== null) {
$content->find('header', 0)->innertext = '';
}
if($content->find('footer', 0) !== null) {
$content->find('footer', 0)->innertext = '';
}
//Remove html nodes, keep only img, links, basic formatting
$content = strip_tags($content, '<a><img><i><u><br><p>');
//Adapt link hrefs: convert relative links into absolute links and bypass external link redirection
$content = preg_replace_callback('/ href=\"([^"]+)\"/i', $unescape_fb_link, $content);
//Clean useless html tag properties and fix link closing tags
foreach (array(
'onmouseover',
'onclick',
'target',
'ajaxify',
'tabindex',
'class',
'style',
'data-[^=]*',
'aria-[^=]*',
'role',
'rel',
'id') as $property_name)
$content = preg_replace('/ ' . $property_name . '=\"[^"]*\"/i', '', $content);
$content = preg_replace('/<\/a [^>]+>/i', '</a>', $content);
//Convert textual representation of emoticons eg
// "<i><u>smile emoticon</u></i>" back to ASCII emoticons eg ":)"
$content = preg_replace_callback('/<i><u>([^ <>]+) ([^<>]+)<\/u><\/i>/i', $unescape_fb_emote, $content);
$item['content'] = $content;
$title = $author;
if (strlen($title) > 24)
$title = substr($title, 0, strpos(wordwrap($title, 24), "\n")) . '...';
$title = $title . ' | ' . strip_tags($content);
if (strlen($title) > 64)
$title = substr($title, 0, strpos(wordwrap($title, 64), "\n")) . '...';
$item['title'] = $title;
$item['author'] = $author;
array_push($this->items, $item);
}
}
// Currently not used. Is used to get more than only 3 elements, as they appear on another page.
private function computeNextLink($string, $pageID){
$regex = implode(
'',
array(
'/timeline_unit',
"\\\\\\\\u00253A1",
"\\\\\\\\u00253A([0-9]*)",
"\\\\\\\\u00253A([0-9]*)",
"\\\\\\\\u00253A([0-9]*)",
"\\\\\\\\u00253A([0-9]*)/"
)
);
preg_match($regex, $string, $result);
return implode(
'',
array(
'https://touch.facebook.com/pages_reaction_units/more/?page_id=',
$pageID,
'&cursor=%7B%22timeline_cursor%22%3A%22timeline_unit%3A1%3A',
$result[1],
'%3A',
$result[2],
'%3A',
$result[3],
'%3A',
$result[4],
'%22%2C%22timeline_section_cursor%22%3A%7B%7D%2C%22',
'has_next_page%22%3Atrue%7D&surface=mobile_page_home&unit_count=3'
)
);
}
//Builds the HTML from the encoded JS that Facebook provides.
private function buildContent($pageContent){
// The html ends with:
// /div>","replaceifexists
$regex = '/\\"html\\":(\".+\/div>"),"replace/';
preg_match($regex, $pageContent, $result);
return str_get_html(html_entity_decode(json_decode($result[1])));
}
//Builds the cookie from the page, as Facebook sometimes refuses to give
//the page if no cookie is provided.
private function getCookies($pageURL){
$ctx = stream_context_create(array(
'http' => array(
'user_agent' => 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0',
'Accept' => 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
)
)
);
$a = file_get_contents($pageURL, 0, $ctx);
//First request to get the cookie
$cookies = '';
foreach($http_response_header as $hdr) {
if(strpos($hdr, 'Set-Cookie') !== false) {
$cLine = explode(':', $hdr)[1];
$cLine = explode(';', $cLine)[0];
$cookies .= ';' . $cLine;
}
}
return substr($cookies, 1);
}
//Get the page ID from the Facebook page.
private function getPageID($page, $cookies){
$context = stream_context_create(array(
'http' => array(
'user_agent' => 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0',
'header' => 'Cookie: ' . $cookies
)
)
);
$pageContent = file_get_contents($page, 0, $context);
if(strpos($pageContent, 'signup-button') != false) {
return -1;
}
//Get the page ID if we don't have a captcha
$regex = '/page_id=([0-9]*)&/';
preg_match($regex, $pageContent, $matches);
if(count($matches) > 0) {
return $matches[1];
}
//Get the page ID if we do have a captcha
$regex = '/"pageID":"([0-9]*)"/';
preg_match($regex, $pageContent, $matches);
return $matches[1];
}
public function getName(){
return (isset($this->name) ? $this->name . ' - ' : '') . 'Facebook Bridge';
}
public function getURI(){
return 'http://facebook.com';
}
public function getCacheDuration(){
return 60 * 60 * 3; // 5 minutes
}
}

54
bridges/FDroidBridge.php Normal file
View File

@@ -0,0 +1,54 @@
<?php
class FDroidBridge extends BridgeAbstract {
const MAINTAINER = 'Mitsukarenai';
const NAME = 'F-Droid Bridge';
const URI = 'https://f-droid.org/';
const CACHE_TIMEOUT = 60 * 60 * 2; // 2 hours
const DESCRIPTION = 'Returns latest added/updated apps on the open-source Android apps repository F-Droid';
const PARAMETERS = array( array(
'u' => array(
'name' => 'Widget selection',
'type' => 'list',
'required' => true,
'values' => array(
'Latest added apps' => 'added',
'Latest updated apps' => 'updated'
)
)
));
public function collectData(){
$url = self::URI;
$html = getSimpleHTMLDOM($url)
or returnServerError('Could not request F-Droid.');
// targetting the corresponding widget based on user selection
// "updated" is the 4th widget on the page, "added" is the 5th
switch($this->getInput('u')) {
case 'updated':
$html_widget = $html->find('div.sidebar-widget', 4);
break;
default:
$html_widget = $html->find('div.sidebar-widget', 5);
break;
}
// and now extracting app info from the selected widget (and yeah turns out icons are of heterogeneous sizes)
foreach($html_widget->find('a') as $element) {
$item = array();
$item['uri'] = self::URI . $element->href;
$item['title'] = $element->find('h4', 0)->plaintext;
$item['icon'] = $element->find('img', 0)->src;
$item['summary'] = $element->find('span.package-summary', 0)->plaintext;
$item['content'] = '
<a href="'.$item['uri'].'">
<img alt="" style="max-height:128px" src="'.$item['icon'].'">
</a><br>'.$item['summary'];
$this->items[] = $item;
}
}
}

View File

@@ -1,46 +1,283 @@
<?php
class FacebookBridge extends BridgeAbstract{
class FacebookBridge extends BridgeAbstract {
const MAINTAINER = "teromene";
const NAME = "Facebook";
const URI = "https://www.facebook.com/";
const MAINTAINER = 'teromene, logmanoriginal';
const NAME = 'Facebook';
const URI = 'https://www.facebook.com/';
const CACHE_TIMEOUT = 300; // 5min
const DESCRIPTION = "Input a page title or a profile log. For a profile log, please insert the parameter as follow : myExamplePage/132621766841117";
const DESCRIPTION = 'Input a page title or a profile log. For a profile log,
please insert the parameter as follow : myExamplePage/132621766841117';
const PARAMETERS =array( array(
'u'=>array(
'name'=>'Username',
'required'=>true
)
));
const PARAMETERS = array(
'User' => array(
'u' => array(
'name' => 'Username',
'required' => true
),
'media_type' => array(
'name' => 'Media type',
'type' => 'list',
'required' => false,
'values' => array(
'All' => 'all',
'Video' => 'video',
'No Video' => 'novideo'
),
'defaultValue' => 'all'
),
'skip_reviews' => array(
'name' => 'Skip reviews',
'type' => 'checkbox',
'required' => false,
'defaultValue' => false,
'title' => 'Feed includes reviews when checked'
)
),
'Group' => array(
'g' => array(
'name' => 'Group',
'type' => 'text',
'required' => true,
'exampleValue' => 'https://www.facebook.com/groups/743149642484225',
'title' => 'Insert group name or facebook group URL'
)
)
);
private $authorName='';
private $authorName = '';
private $groupName = '';
public function collectData(){
public function getURI() {
$uri = self::URI;
switch($this->queriedContext) {
case 'Group':
$uri .= 'groups/' . $this->sanitizeGroup(filter_var($this->getInput('g'), FILTER_SANITIZE_URL));
break;
}
return $uri .= '?_fb_noscript=1';
}
public function collectData() {
switch($this->queriedContext) {
case 'Group':
$this->collectGroupData();
break;
case 'User':
$this->collectUserData();
break;
default:
returnClientError('Unknown context: "' . $this->queriedContext . '"!');
}
}
#region Group
private function collectGroupData() {
$header = array('Accept-Language: ' . getEnv('HTTP_ACCEPT_LANGUAGE') . "\r\n");
$html = getSimpleHTMLDOM($this->getURI(), $header)
or returnServerError('Failed loading facebook page: ' . $this->getURI());
if(!$this->isPublicGroup($html)) {
returnClientError('This group is not public! RSS-Bridge only supports public groups!');
}
defaultLinkTo($html, substr(self::URI, 0, strlen(self::URI) - 1));
$this->groupName = $this->extractGroupName($html);
$posts = $html->find('div.userContentWrapper')
or returnServerError('Failed finding posts!');
foreach($posts as $post) {
$item = array();
$item['uri'] = $this->extractGroupURI($post);
$item['title'] = $this->extractGroupTitle($post);
$item['author'] = $this->extractGroupAuthor($post);
$item['content'] = $this->extractGroupContent($post);
$item['timestamp'] = $this->extractGroupTimestamp($post);
$item['enclosures'] = $this->extractGroupEnclosures($post);
$this->items[] = $item;
}
}
private function sanitizeGroup($group) {
if(filter_var(
$group,
FILTER_VALIDATE_URL,
FILTER_FLAG_HOST_REQUIRED | FILTER_FLAG_PATH_REQUIRED)) {
// User provided a URL
$urlparts = parse_url($group);
if($urlparts['host'] !== parse_url(self::URI)['host']
&& 'www.' . $urlparts['host'] !== parse_url(self::URI)['host']) {
returnClientError('The host you provided is invalid! Received "'
. $urlparts['host']
. '", expected "'
. parse_url(self::URI)['host']
. '"!');
}
return explode('/', $urlparts['path'])[2];
} elseif(strpos($group, '/') !== false) {
returnClientError('The group you provided is invalid: ' . $group);
} else {
return $group;
}
}
private function isPublicGroup($html) {
// Facebook redirects to the groups about page for non-public groups
$about = $html->find('#pagelet_group_about', 0);
return !($about);
}
private function extractGroupName($html) {
$ogtitle = $html->find('meta[property="og:title"]', 0)
or returnServerError('Unable to find group title!');
return htmlspecialchars_decode($ogtitle->content, ENT_QUOTES);
}
private function extractGroupURI($post) {
$elements = $post->find('a')
or returnServerError('Unable to find URI!');
foreach($elements as $anchor) {
// Find the one that is a permalink
if(strpos($anchor->href, 'permalink') !== false) {
return $anchor->href;
}
}
return null;
}
private function extractGroupContent($post) {
$content = $post->find('div.userContent', 0)
or returnServerError('Unable to find user content!');
return $content->innertext . $content->next_sibling()->innertext;
}
private function extractGroupTimestamp($post) {
$element = $post->find('abbr[data-utime]', 0)
or returnServerError('Unable to find timestamp!');
return $element->getAttribute('data-utime');
}
private function extractGroupAuthor($post) {
$element = $post->find('img', 0)
or returnServerError('Unable to find author information!');
return $element->{'aria-label'};
}
private function extractGroupEnclosures($post) {
$elements = $post->find('div.userContent', 0)->next_sibling()->find('img');
$enclosures = array();
foreach($elements as $enclosure) {
$enclosures[] = $enclosure->src;
}
return empty($enclosures) ? null : $enclosures;
}
private function extractGroupTitle($post) {
$element = $post->find('h5', 0)
or returnServerError('Unable to find title!');
if(strpos($element->plaintext, 'shared') === false) {
$content = strip_tags($this->extractGroupContent($post));
return $this->extractGroupAuthor($post)
. ' posted: '
. substr(
$content,
0,
strpos(wordwrap($content, 64), "\n")
)
. '...';
}
return $element->plaintext;
}
#endregion
private function collectUserData(){
//Extract a string using start and end delimiters
function ExtractFromDelimiters($string, $start, $end) {
if (strpos($string, $start) !== false) {
function extractFromDelimiters($string, $start, $end){
if(strpos($string, $start) !== false) {
$section_retrieved = substr($string, strpos($string, $start) + strlen($start));
$section_retrieved = substr($section_retrieved, 0, strpos($section_retrieved, $end));
return $section_retrieved;
} return false;
}
return false;
}
//Utility function for cleaning a Facebook link
$unescape_fb_link = function ($matches) {
if (is_array($matches) && count($matches) > 1) {
$unescape_fb_link = function($matches){
if(is_array($matches) && count($matches) > 1) {
$link = $matches[1];
if (strpos($link, '/') === 0)
$link = self::URI.$link.'"';
if (strpos($link, 'facebook.com/l.php?u=') !== false)
$link = urldecode(ExtractFromDelimiters($link, 'facebook.com/l.php?u=', '&'));
return ' href="'.$link.'"';
if(strpos($link, '/') === 0)
$link = self::URI . $link;
if(strpos($link, 'facebook.com/l.php?u=') !== false)
$link = urldecode(extractFromDelimiters($link, 'facebook.com/l.php?u=', '&'));
return ' href="' . $link . '"';
}
};
//Utility function for converting facebook emoticons
$unescape_fb_emote = function ($matches) {
$unescape_fb_emote = function($matches){
static $facebook_emoticons = array(
'smile' => ':)',
'frown' => ':(',
@@ -75,26 +312,26 @@ class FacebookBridge extends BridgeAbstract{
$html = null;
//Handle captcha response sent by the viewer
if (isset($_POST['captcha_response']))
{
if (isset($_POST['captcha_response'])) {
if (session_status() == PHP_SESSION_NONE)
session_start();
if (isset($_SESSION['captcha_fields'], $_SESSION['captcha_action']))
{
if (isset($_SESSION['captcha_fields'], $_SESSION['captcha_action'])) {
$captcha_action = $_SESSION['captcha_action'];
$captcha_fields = $_SESSION['captcha_fields'];
$captcha_fields['captcha_response'] = preg_replace("/[^a-zA-Z0-9]+/", "", $_POST['captcha_response']);
$http_options = array(
'http' => array(
'method' => 'POST',
'user_agent'=> ini_get('user_agent'),
'header'=>array("Content-type: application/x-www-form-urlencoded\r\nReferer: $captcha_action\r\nCookie: noscript=1\r\n"),
'content' => http_build_query($captcha_fields),
),
$captcha_fields['captcha_response'] = preg_replace('/[^a-zA-Z0-9]+/', '', $_POST['captcha_response']);
$header = array("Content-type:
application/x-www-form-urlencoded\r\nReferer: $captcha_action\r\nCookie: noscript=1\r\n");
$opts = array(
CURLOPT_POST => 1,
CURLOPT_POSTFIELDS => http_build_query($captcha_fields)
);
$context = stream_context_create($http_options);
$html = getContents($captcha_action, false, $context);
if ($html === FALSE) { returnServerError('Failed to submit captcha response back to Facebook'); }
$html = getContents($captcha_action, $header, $opts);
if($html === false) {
returnServerError('Failed to submit captcha response back to Facebook');
}
unset($_SESSION['captcha_fields']);
$html = str_get_html($html);
}
@@ -103,20 +340,53 @@ class FacebookBridge extends BridgeAbstract{
}
//Retrieve page contents
if (is_null($html)) {
if (!strpos($this->getInput('u'), "/")) {
$html = getSimpleHTMLDOM(self::URI.urlencode($this->getInput('u')).'?_fb_noscript=1')
or returnServerError('No results for this query.');
if(is_null($html)) {
$header = array('Accept-Language: ' . getEnv('HTTP_ACCEPT_LANGUAGE') . "\r\n");
// Check if the user provided a fully qualified URL
if (filter_var($this->getInput('u'), FILTER_VALIDATE_URL)) {
$urlparts = parse_url($this->getInput('u'));
if($urlparts['host'] !== parse_url(self::URI)['host']) {
returnClientError('The host you provided is invalid! Received "'
. $urlparts['host']
. '", expected "'
. parse_url(self::URI)['host']
. '"!');
}
if(!array_key_exists('path', $urlparts)
|| $urlparts['path'] === '/') {
returnClientError('The URL you provided doesn\'t contain the user name!');
}
$user = explode('/', $urlparts['path'])[1];
$html = getSimpleHTMLDOM(self::URI . urlencode($user) . '?_fb_noscript=1', $header)
or returnServerError('No results for this query.');
} else {
$html = getSimpleHTMLDOM(self::URI.'pages/'.$this->getInput('u').'?_fb_noscript=1')
or returnServerError('No results for this query.');
// First character cannot be a forward slash
if(strpos($this->getInput('u'), '/') === 0) {
returnClientError('Remove leading slash "/" from the username!');
}
if(!strpos($this->getInput('u'), '/')) {
$html = getSimpleHTMLDOM(self::URI . urlencode($this->getInput('u')) . '?_fb_noscript=1', $header)
or returnServerError('No results for this query.');
} else {
$html = getSimpleHTMLDOM(self::URI . 'pages/' . $this->getInput('u') . '?_fb_noscript=1', $header)
or returnServerError('No results for this query.');
}
}
}
//Handle captcha form?
$captcha = $html->find('div.captcha_interstitial', 0);
if (!is_null($captcha))
{
if (!is_null($captcha)) {
//Save form for submitting after getting captcha response
if (session_status() == PHP_SESSION_NONE)
session_start();
@@ -124,87 +394,182 @@ class FacebookBridge extends BridgeAbstract{
foreach ($captcha->find('input, button') as $input)
$captcha_fields[$input->name] = $input->value;
$_SESSION['captcha_fields'] = $captcha_fields;
$_SESSION['captcha_action'] = self::URI.$captcha->find('form', 0)->action;
$_SESSION['captcha_action'] = $captcha->find('form', 0)->action;
//Show captcha filling form to the viewer, proxying the captcha image
$img = base64_encode(getContents($captcha->find('img', 0)->src));
header('HTTP/1.1 500 '.Http::getMessageForCode(500));
http_response_code(500);
header('Content-Type: text/html');
die('<form method="post" action="?'.$_SERVER['QUERY_STRING'].'">'
.'<h2>Facebook captcha challenge</h2>'
.'<p>Unfortunately, rss-bridge cannot fetch the requested page.<br />'
.'Facebook wants rss-bridge to resolve the following captcha:</p>'
.'<p><img src="data:image/png;base64,'.$img.'" /></p>'
.'<p><b>Response:</b> <input name="captcha_response" placeholder="please fill in" />'
.'<input type="submit" value="Submit!" /></p>'
.'</form>');
$message = <<<EOD
<form method="post" action="?{$_SERVER['QUERY_STRING']}">
<h2>Facebook captcha challenge</h2>
<p>Unfortunately, rss-bridge cannot fetch the requested page.<br />
Facebook wants rss-bridge to resolve the following captcha:</p>
<p><img src="data:image/png;base64,{$img}" /></p>
<p><b>Response:</b> <input name="captcha_response" placeholder="please fill in" />
<input type="submit" value="Submit!" /></p>
</form>
EOD;
die($message);
}
//No captcha? We can carry on retrieving page contents :)
$element = $html->find('#pagelet_timeline_main_column')[0]->children(0)->children(0)->children(0)->next_sibling()->children(0);
//First, we check wether the page is public or not
$loginForm = $html->find('._585r', 0);
if($loginForm != null) {
returnServerError('You must be logged in to view this page. This is not supported by RSS-Bridge.');
}
$element = $html
->find('#pagelet_timeline_main_column')[0]
->children(0)
->children(0)
->children(0)
->next_sibling()
->children(0);
if(isset($element)) {
$author = str_replace(' | Facebook', '', $html->find('title#pageTitle', 0)->innertext);
$profilePic = 'https://graph.facebook.com/'.$this->getInput('u').'/picture?width=200&amp;height=200';
$profilePic = 'https://graph.facebook.com/'
. $this->getInput('u')
. '/picture?width=200&amp;height=200';
$this->authorName = $author;
foreach($element->children() as $post) {
foreach($element->children() as $cell) {
// Manage summary posts
if(strpos($cell->class, '_3xaf') !== false) {
$posts = $cell->children();
} else {
$posts = array($cell);
}
$item = array();
// Optionally skip reviews
if($this->getInput('skip_reviews')
&& !is_null($cell->find('#review_composer_container', 0))) {
continue;
}
if (count($post->find('abbr')) > 0) {
//Retrieve post contents
$content = preg_replace('/(?i)><div class=\"clearfix([^>]+)>(.+?)div\ class=\"userContent\"/i', '', $post);
$content = preg_replace('/(?i)><div class=\"_59tj([^>]+)>(.+?)<\/div><\/div><a/i', '', $content);
$content = preg_replace('/(?i)><div class=\"_3dp([^>]+)>(.+?)div\ class=\"[^u]+userContent\"/i', '', $content);
$content = preg_replace('/(?i)><div class=\"_4l5([^>]+)>(.+?)<\/div>/i', '', $content);
//Remove html nodes, keep only img, links, basic formatting
$content = strip_tags($content,'<a><img><i><u>');
//Adapt link hrefs: convert relative links into absolute links and bypass external link redirection
$content = preg_replace_callback('/ href=\"([^"]+)\"/i', $unescape_fb_link, $content);
//Clean useless html tag properties and fix link closing tags
foreach (array('onmouseover', 'onclick', 'target', 'ajaxify', 'tabindex',
'class', 'style', 'data-[^=]*', 'aria-[^=]*', 'role', 'rel', 'id') as $property_name)
$content = preg_replace('/ '.$property_name.'=\"[^"]*\"/i', '', $content);
$content = preg_replace('/<\/a [^>]+>/i', '</a>', $content);
//Convert textual representation of emoticons eg "<i><u>smile emoticon</u></i>" back to ASCII emoticons eg ":)"
$content = preg_replace_callback('/<i><u>([^ <>]+) ([^<>]+)<\/u><\/i>/i', $unescape_fb_emote, $content);
//Retrieve date of the post
$date = $post->find("abbr")[0];
if(isset($date) && $date->hasAttribute('data-utime')) {
$date = $date->getAttribute('data-utime');
} else {
$date = 0;
foreach($posts as $post) {
// Check media type
switch($this->getInput('media_type')) {
case 'all': break;
case 'video':
if(empty($post->find('[aria-label=Video]'))) continue 2;
break;
case 'novideo':
if(!empty($post->find('[aria-label=Video]'))) continue 2;
break;
default: break;
}
//Build title from username and content
$title = $author;
if (strlen($title) > 24)
$title = substr($title, 0, strpos(wordwrap($title, 24), "\n")).'...';
$title = $title.' | '.strip_tags($content);
if (strlen($title) > 64)
$title = substr($title, 0, strpos(wordwrap($title, 64), "\n")).'...';
$item = array();
//Build and add final item
$item['uri'] = self::URI.$post->find('abbr')[0]->parent()->getAttribute('href');
$item['content'] = $content;
$item['title'] = $title;
$item['author'] = $author;
$item['timestamp'] = $date;
$this->items[] = $item;
if(count($post->find('abbr')) > 0) {
//Retrieve post contents
$content = preg_replace(
'/(?i)><div class=\"clearfix([^>]+)>(.+?)div\ class=\"userContent\"/i',
'',
$post);
$content = preg_replace(
'/(?i)><div class=\"_59tj([^>]+)>(.+?)<\/div><\/div><a/i',
'',
$content);
$content = preg_replace(
'/(?i)><div class=\"_3dp([^>]+)>(.+?)div\ class=\"[^u]+userContent\"/i',
'',
$content);
$content = preg_replace(
'/(?i)><div class=\"_4l5([^>]+)>(.+?)<\/div>/i',
'',
$content);
//Remove html nodes, keep only img, links, basic formatting
$content = strip_tags($content, '<a><img><i><u><br><p>');
//Adapt link hrefs: convert relative links into absolute links and bypass external link redirection
$content = preg_replace_callback('/ href=\"([^"]+)\"/i', $unescape_fb_link, $content);
//Clean useless html tag properties and fix link closing tags
foreach (array(
'onmouseover',
'onclick',
'target',
'ajaxify',
'tabindex',
'class',
'style',
'data-[^=]*',
'aria-[^=]*',
'role',
'rel',
'id') as $property_name)
$content = preg_replace('/ ' . $property_name . '=\"[^"]*\"/i', '', $content);
$content = preg_replace('/<\/a [^>]+>/i', '</a>', $content);
//Convert textual representation of emoticons eg
//"<i><u>smile emoticon</u></i>" back to ASCII emoticons eg ":)"
$content = preg_replace_callback(
'/<i><u>([^ <>]+) ([^<>]+)<\/u><\/i>/i',
$unescape_fb_emote,
$content
);
//Retrieve date of the post
$date = $post->find('abbr')[0];
if(isset($date) && $date->hasAttribute('data-utime')) {
$date = $date->getAttribute('data-utime');
} else {
$date = 0;
}
//Build title from username and content
$title = $author;
if(strlen($title) > 24)
$title = substr($title, 0, strpos(wordwrap($title, 24), "\n")) . '...';
$title = $title . ' | ' . strip_tags($content);
if(strlen($title) > 64)
$title = substr($title, 0, strpos(wordwrap($title, 64), "\n")) . '...';
$uri = self::URI . $post->find('abbr')[0]->parent()->getAttribute('href');
//Build and add final item
$item['uri'] = htmlspecialchars_decode($uri);
$item['content'] = htmlspecialchars_decode($content);
$item['title'] = $title;
$item['author'] = $author;
$item['timestamp'] = $date;
$this->items[] = $item;
}
}
}
}
}
public function getName() {
return (isset($this->authorName) ? $this->authorName.' - ' : '').'Facebook Bridge';
public function getName(){
switch($this->queriedContext) {
case 'User':
if(!empty($this->authorName)) {
return isset($this->extraInfos['name']) ? $this->extraInfos['name'] : $this->authorName
. ' - Facebook Bridge';
}
break;
case 'Group':
if(!empty($this->groupName)) {
return $this->groupName . ' - Facebook Bridge';
}
break;
}
return parent::getName();
}
}

View File

@@ -1,62 +1,62 @@
<?php
class FeedExpanderExampleBridge extends FeedExpander {
const MAINTAINER = 'logmanoriginal';
const NAME = 'FeedExpander Example';
const URI = '#';
const DESCRIPTION = 'Example bridge to test FeedExpander';
const MAINTAINER = 'logmanoriginal';
const NAME = 'FeedExpander Example';
const URI = '#';
const DESCRIPTION = 'Example bridge to test FeedExpander';
const PARAMETERS = array(
'Feed' => array(
'version' => array(
'name' => 'Version',
'type' => 'list',
'required' => true,
'title' => 'Select your feed format/version',
'defaultValue' => 'RSS 2.0',
'values' => array(
'RSS 0.91' => 'rss_0_9_1',
'RSS 1.0' => 'rss_1_0',
'RSS 2.0' => 'rss_2_0',
'ATOM 1.0' => 'atom_1_0'
)
)
)
);
const PARAMETERS = array(
'Feed' => array(
'version' => array(
'name' => 'Version',
'type' => 'list',
'required' => true,
'title' => 'Select your feed format/version',
'defaultValue' => 'RSS 2.0',
'values' => array(
'RSS 0.91' => 'rss_0_9_1',
'RSS 1.0' => 'rss_1_0',
'RSS 2.0' => 'rss_2_0',
'ATOM 1.0' => 'atom_1_0'
)
)
)
);
public function collectData(){
switch($this->getInput('version')){
case 'rss_0_9_1':
parent::collectExpandableDatas('http://static.userland.com/gems/backend/sampleRss.xml');
break;
case 'rss_1_0':
parent::collectExpandableDatas('http://feeds.nature.com/nature/rss/current?format=xml');
break;
case 'rss_2_0':
parent::collectExpandableDatas('http://feeds.rssboard.org/rssboard?format=xml');
break;
case 'atom_1_0':
parent::collectExpandableDatas('http://segfault.linuxmint.com/feed/atom/');
break;
default: returnClientError('Unknown version ' . $this->getInput('version') . '!');
}
}
public function collectData(){
switch($this->getInput('version')) {
case 'rss_0_9_1':
parent::collectExpandableDatas('http://static.userland.com/gems/backend/sampleRss.xml');
break;
case 'rss_1_0':
parent::collectExpandableDatas('http://feeds.nature.com/nature/rss/current?format=xml');
break;
case 'rss_2_0':
parent::collectExpandableDatas('http://feeds.rssboard.org/rssboard?format=xml');
break;
case 'atom_1_0':
parent::collectExpandableDatas('http://segfault.linuxmint.com/feed/atom/');
break;
default: returnClientError('Unknown version ' . $this->getInput('version') . '!');
}
}
protected function parseItem($newsItem) {
switch($this->getInput('version')){
case 'rss_0_9_1':
return $this->parseRSS_0_9_1_Item($newsItem);
break;
case 'rss_1_0':
return $this->parseRSS_1_0_Item($newsItem);
break;
case 'rss_2_0':
return $this->parseRSS_2_0_Item($newsItem);
break;
case 'atom_1_0':
return $this->parseATOMItem($newsItem);
break;
default: returnClientError('Unknown version ' . $this->getInput('version') . '!');
}
}
protected function parseItem($newsItem) {
switch($this->getInput('version')) {
case 'rss_0_9_1':
return $this->parseRSS_0_9_1_Item($newsItem);
break;
case 'rss_1_0':
return $this->parseRSS_1_0_Item($newsItem);
break;
case 'rss_2_0':
return $this->parseRSS_2_0_Item($newsItem);
break;
case 'atom_1_0':
return $this->parseATOMItem($newsItem);
break;
default: returnClientError('Unknown version ' . $this->getInput('version') . '!');
}
}
}

View File

@@ -1,23 +1,24 @@
<?php
class FierPandaBridge extends BridgeAbstract {
const MAINTAINER = "snroki";
const NAME = "Fier Panda Bridge";
const URI = "http://www.fier-panda.fr/";
const CACHE_TIMEOUT = 21600; // 6h
const DESCRIPTION = "Returns latest articles from Fier Panda.";
const MAINTAINER = 'snroki';
const NAME = 'Fier Panda Bridge';
const URI = 'http://www.fier-panda.fr/';
const CACHE_TIMEOUT = 21600; // 6h
const DESCRIPTION = 'Returns latest articles from Fier Panda.';
public function collectData(){
$html = getSimpleHTMLDOM(self::URI) or returnServerError('Could not request Fier Panda.');
public function collectData(){
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not request Fier Panda.');
foreach($html->find('div.container-content article') as $element) {
$item = array();
$item['uri'] = $this->getURI().$element->find('a', 0)->href;
$item['title'] = trim($element->find('h1 a', 0)->innertext);
// Remove the link at the end of the article
$element->find('p a', 0)->outertext = '';
$item['content'] = $element->find('p', 0)->innertext;
$this->items[] = $item;
}
}
foreach($html->find('div.container-content article') as $element) {
$item = array();
$item['uri'] = $this->getURI() . $element->find('a', 0)->href;
$item['title'] = trim($element->find('h1 a', 0)->innertext);
// Remove the link at the end of the article
$element->find('p a', 0)->outertext = '';
$item['content'] = $element->find('p', 0)->innertext;
$this->items[] = $item;
}
}
}

77
bridges/FilterBridge.php Normal file
View File

@@ -0,0 +1,77 @@
<?php
class FilterBridge extends FeedExpander {
const MAINTAINER = 'Frenzie';
const NAME = 'Filter';
const CACHE_TIMEOUT = 3600; // 1h
const DESCRIPTION = 'Filters a feed of your choice';
const PARAMETERS = array(array(
'url' => array(
'name' => 'Feed URL',
'required' => true,
),
'filter' => array(
'name' => 'Filter item title (regular expression)',
'required' => false,
),
'filter_type' => array(
'name' => 'Filter type',
'type' => 'list',
'required' => false,
'values' => array(
'Permit' => 'permit',
'Block' => 'block',
),
'defaultValue' => 'permit',
),
));
protected function parseItem($newItem){
$item = parent::parseItem($newItem);
switch(true) {
case $this->getFilterType() === 'permit':
if (preg_match($this->getFilter(), $item['title'])) {
return $item;
}
break;
case $this->getFilterType() === 'block':
if (!preg_match($this->getFilter(), $item['title'])) {
return $item;
}
break;
}
return null;
}
protected function getFilter(){
return '/' . $this->getInput('filter') . '/';
}
protected function getFilterType(){
return $this->getInput('filter_type');
}
public function getURI(){
$url = $this->getInput('url');
if(empty($url)) {
$url = parent::getURI();
}
return $url;
}
public function collectData(){
if($this->getInput('url') && substr($this->getInput('url'), 0, strlen('http')) !== 'http') {
// just in case someone find a way to access local files by playing with the url
returnClientError('The url parameter must either refer to http or https protocol.');
}
try{
$this->collectExpandableDatas($this->getURI());
} catch (HttpException $e) {
$this->collectExpandableDatas($this->getURI());
}
}
}

120
bridges/FlickrBridge.php Normal file
View File

@@ -0,0 +1,120 @@
<?php
/* This is a mashup of FlickrExploreBridge by sebsauvage and FlickrTagBridge
* by erwang, providing the functionality of both in one.
*/
class FlickrBridge extends BridgeAbstract {
const MAINTAINER = 'logmanoriginal';
const NAME = 'Flickr Bridge';
const URI = 'https://www.flickr.com/';
const CACHE_TIMEOUT = 21600; // 6 hours
const DESCRIPTION = 'Returns images from Flickr';
const PARAMETERS = array(
'Explore' => array(),
'By keyword' => array(
'q' => array(
'name' => 'Keyword',
'type' => 'text',
'required' => true,
'title' => 'Insert keyword',
'exampleValue' => 'bird'
)
),
'By username' => array(
'u' => array(
'name' => 'Username',
'type' => 'text',
'required' => true,
'title' => 'Insert username (as shown in the address bar)',
'exampleValue' => 'flickr'
)
),
);
public function collectData(){
switch($this->queriedContext) {
case 'Explore':
$key = 'photos';
$html = getSimpleHTMLDOM(self::URI . 'explore')
or returnServerError('Could not request Flickr.');
break;
case 'By keyword':
$key = 'photos';
$html = getSimpleHTMLDOM(self::URI . 'search/?q=' . urlencode($this->getInput('q')) . '&s=rec')
or returnServerError('No results for this query.');
break;
case 'By username':
$key = 'photoPageList';
$html = getSimpleHTMLDOM(self::URI . 'photos/' . urlencode($this->getInput('u')))
or returnServerError('Requested username can\'t be found.');
break;
default:
returnClientError('Invalid context: ' . $this->queriedContext);
}
// Find SCRIPT containing JSON data
$model = $html->find('.modelExport', 0);
$model_text = $model->innertext;
// Find start and end of JSON data
$start = strpos($model_text, 'modelExport:') + strlen('modelExport:');
$end = strpos($model_text, 'auth:') - strlen('auth:');
// Dissect JSON data and remove trailing comma
$model_text = trim(substr($model_text, $start, $end - $start));
$model_text = substr($model_text, 0, strlen($model_text) - 1);
$model_json = json_decode($model_text, true);
foreach($html->find('.photo-list-photo-view') as $element) {
// Get the styles
$style = explode(';', $element->style);
// Get the background-image style
$backgroundImage = explode(':', end($style));
// URI type : url(//cX.staticflickr.com/X/XXXXX/XXXXXXXXX.jpg)
$imageURI = trim(str_replace(['url(', ')'], '', end($backgroundImage)));
// Get the image ID
$imageURIs = explode('_', basename($imageURI));
$imageID = reset($imageURIs);
// Use JSON data to build items
foreach(reset($model_json)[0][$key]['_data'] as $element) {
if($element['id'] === $imageID) {
$item = array();
/* Author name depends on scope. On a keyword search the
* author is part of the picture data. On a username search
* the author is part of the owner data.
*/
if(array_key_exists('username', $element)) {
$item['author'] = $element['username'];
} elseif (array_key_exists('owner', reset($model_json)[0])) {
$item['author'] = reset($model_json)[0]['owner']['username'];
}
$item['title'] = (array_key_exists('title', $element) ? $element['title'] : 'Untitled');
$item['uri'] = self::URI . 'photo.gne?id=' . $imageID;
$description = (array_key_exists('description', $element) ? $element['description'] : '');
$item['content'] = '<a href="'
. $item['uri']
. '"><img src="'
. $imageURI
. '" /></a><br><p>'
. $description
. '</p>';
$this->items[] = $item;
break;
}
}
}
}
}

View File

@@ -1,42 +0,0 @@
<?php
class FlickrExploreBridge extends BridgeAbstract{
const MAINTAINER = "sebsauvage";
const NAME = "Flickr Explore";
const URI = "https://www.flickr.com/";
const CACHE_TIMEOUT = 21600; // 6
const DESCRIPTION = "Returns the latest interesting images from Flickr";
public function collectData(){
$html = getSimpleHTMLDOM(self::URI.'explore')
or returnServerError('Could not request Flickr.');
foreach($html->find('.photo-list-photo-view') as $element) {
// Get the styles
$style = explode(';', $element->style);
// Get the background-image style
$backgroundImage = explode(':', end($style));
// URI type : url(//cX.staticflickr.com/X/XXXXX/XXXXXXXXX.jpg)
$imageURI = trim(str_replace(['url(', ')'], '', end($backgroundImage)));
// Get the image ID
$imageURIs = explode('_', basename($imageURI));
$imageID = reset($imageURIs);
// Get the image JSON via Flickr API
$imageJSON = json_decode(getContents(
'https://api.flickr.com/services/rest/?'
.'method=flickr.photos.getInfo&'
.'api_key=103b574d49bd51f0e18bfe907da44a0f&'
.'photo_id='.$imageID.'&'
.'format=json&'
.'nojsoncallback=1'
)) or returnServerError('Could not request Flickr.'); // FIXME: Request time too long...
$item = array();
$item['uri'] = self::URI.'photo.gne?id='.$imageID;
$item['content'] = '<a href="' . $item['uri'] . '"><img src="' . $imageURI . '" /></a>'; // FIXME: Filter javascript ?
$item['title'] = $imageJSON->photo->title->_content;
$this->items[] = $item;
}
}
}

View File

@@ -1,48 +0,0 @@
<?php
class FlickrTagBridge extends BridgeAbstract{
const MAINTAINER = "erwang";
const NAME = "Flickr TagUser";
const URI = "http://www.flickr.com/";
const CACHE_TIMEOUT = 21600; //6h
const DESCRIPTION = "Returns the tagged or user images from Flickr";
const PARAMETERS = array(
'By keyword' => array(
'q'=>array(
'name'=>'keyword',
'required'=>true
)
),
'By username' => array(
'u'=>array(
'name'=>'Username',
'required'=>true
)
),
);
public function collectData(){
switch($this->queriedContext){
case 'By keyword':
$html = getSimpleHTMLDOM(self::URI.'search/?q='.urlencode($this->getInput('q')).'&s=rec')
or returnServerError('No results for this query.');
break;
case 'by username':
$html = getSimpleHTMLDOM(self::URI.'photos/'.urlencode($this->getInput('u')).'/')
or returnServerError('Requested username can\'t be found.');
break;
}
foreach($html->find('span.photo_container') as $element) {
$item = array();
$item['uri'] = self::URI.$element->find('a',0)->href;
$thumbnailUri = $element->find('img',0)->getAttribute('data-defer-src');
$item['content'] = '<a href="' . $item['uri'] . '"><img src="' . $thumbnailUri . '" /></a>'; // FIXME: Filter javascript ?
$item['title'] = $element->find('a',0)->title;
$this->items[] = $item;
}
}
}

View File

@@ -1,40 +1,75 @@
<?php
class FootitoBridge extends BridgeAbstract{
class FootitoBridge extends BridgeAbstract {
const MAINTAINER = "superbaillot.net";
const NAME = "Footito";
const URI = "http://www.footito.fr/";
const DESCRIPTION = "Footito";
const MAINTAINER = 'superbaillot.net';
const NAME = 'Footito';
const URI = 'http://www.footito.fr/';
const DESCRIPTION = 'Footito';
public function collectData(){
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not request Footito.');
public function collectData(){
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not request Footito.');
foreach($html->find('div.post') as $element) {
$item = array();
foreach($html->find('div.post') as $element) {
$item = array();
$content = trim($element->innertext);
$content = str_replace("<img", "<img style='float : left;'", $content );
$content = str_replace("class=\"logo\"", "style='float : left;'", $content );
$content = str_replace("class=\"contenu\"", "style='margin-left : 60px;'", $content );
$content = str_replace("class=\"responsive-comment\"", "style='border-top : 1px #DDD solid; background-color : white; padding : 10px;'", $content );
$content = str_replace("class=\"jaime\"", "style='display : none;'", $content );
$content = str_replace("class=\"auteur-event responsive\"", "style='display : none;'", $content );
$content = str_replace("class=\"report-abuse-button\"", "style='display : none;'", $content );
$content = str_replace("class=\"reaction clearfix\"", "style='margin : 10px 0px; padding : 5px; border-bottom : 1px #DDD solid;'", $content );
$content = str_replace("class=\"infos\"", "style='font-size : 0.7em;'", $content );
$content = trim($element->innertext);
$content = str_replace(
'<img',
"<img style='float : left;'",
$content );
$item['content'] = $content;
$content = str_replace(
'class="logo"',
"style='float : left;'",
$content );
$title = $element->find('.contenu .texte ', 0)->plaintext;
$item['title'] = $title;
$content = str_replace(
'class="contenu"',
"style='margin-left : 60px;'",
$content );
$info = $element->find('div.infos', 0);
$content = str_replace(
'class="responsive-comment"',
"style='border-top : 1px #DDD solid; background-color : white; padding : 10px;'",
$content );
$item['timestamp'] = strtotime($info->find('time', 0)->datetime);
$item['author'] = $info->find('a.auteur', 0)->plaintext;
$content = str_replace(
'class="jaime"',
"style='display : none;'",
$content );
$this->items[] = $item;
}
}
$content = str_replace(
'class="auteur-event responsive"',
"style='display : none;'",
$content );
$content = str_replace(
'class="report-abuse-button"',
"style='display : none;'",
$content );
$content = str_replace(
'class="reaction clearfix"',
"style='margin : 10px 0px; padding : 5px; border-bottom : 1px #DDD solid;'",
$content );
$content = str_replace(
'class="infos"',
"style='font-size : 0.7em;'",
$content );
$item['content'] = $content;
$title = $element->find('.contenu .texte ', 0)->plaintext;
$item['title'] = $title;
$info = $element->find('div.infos', 0);
$item['timestamp'] = strtotime($info->find('time', 0)->datetime);
$item['author'] = $info->find('a.auteur', 0)->plaintext;
$this->items[] = $item;
}
}
}

View File

@@ -1,68 +1,78 @@
<?php
class FourchanBridge extends BridgeAbstract{
class FourchanBridge extends BridgeAbstract {
const MAINTAINER = "mitsukarenai";
const NAME = "4chan";
const URI = "https://boards.4chan.org/";
const MAINTAINER = 'mitsukarenai';
const NAME = '4chan';
const URI = 'https://boards.4chan.org/';
const CACHE_TIMEOUT = 300; // 5min
const DESCRIPTION = "Returns posts from the specified thread";
const DESCRIPTION = 'Returns posts from the specified thread';
const PARAMETERS = array( array(
'c'=>array(
'name'=>'Thread category',
'required'=>true
),
't'=>array(
'name'=>'Thread number',
'type'=>'number',
'required'=>true
)
));
const PARAMETERS = array( array(
'c' => array(
'name' => 'Thread category',
'required' => true
),
't' => array(
'name' => 'Thread number',
'type' => 'number',
'required' => true
)
));
public function getURI(){
return static::URI.$this->getInput('c').'/thread/'.$this->getInput('t');
}
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError("Could not request 4chan, thread not found");
foreach($html->find('div.postContainer') as $element) {
$item = array();
$item['id'] = $element->find('.post', 0)->getAttribute('id');
$item['uri'] = $this->getURI().'#'.$item['id'];
$item['timestamp'] = $element->find('span.dateTime', 0)->getAttribute('data-utc');
$item['author'] = $element->find('span.name', 0)->plaintext;
$file=$element->find('.file', 0);
if(!empty($file) ) {
$item['image'] = $element->find('.file a', 0)->href;
$item['imageThumb'] = $element->find('.file img', 0)->src;
if(!isset($item['imageThumb']) and strpos($item['image'], '.swf') !== FALSE)
$item['imageThumb'] = 'http://i.imgur.com/eO0cxf9.jpg';
}
if(!empty($element->find('span.subject', 0)->innertext )) {
$item['subject'] = $element->find('span.subject', 0)->innertext;
public function getURI(){
if(!is_null($this->getInput('c')) && !is_null($this->getInput('t'))) {
return static::URI . $this->getInput('c') . '/thread/' . $this->getInput('t');
}
$item['title'] = 'reply '.$item['id'].' | '.$item['author'];
if(isset($item['subject'])){
$item['title'] = $item['subject'].' - '.$item['title'];
}
$content = $element->find('.postMessage', 0)->innertext;
$content = str_replace('href="#p','href="'.$this->getURI().'#p',$content);
$item['content'] = '<span id="'.$item['id'].'">'.$content.'</span>';
if(isset($item['image'])){
$item['content'] = '<a href="'.$item['image'].'">'
.'<img alt="'.$item['id'].'" src="'.$item['imageThumb'].'" />'
.'</a><br>'
.$item['content'];
}
$this->items[] = $item;
return parent::getURI();
}
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request 4chan, thread not found');
foreach($html->find('div.postContainer') as $element) {
$item = array();
$item['id'] = $element->find('.post', 0)->getAttribute('id');
$item['uri'] = $this->getURI() . '#' . $item['id'];
$item['timestamp'] = $element->find('span.dateTime', 0)->getAttribute('data-utc');
$item['author'] = $element->find('span.name', 0)->plaintext;
$file = $element->find('.file', 0);
if(!empty($file)) {
$item['image'] = $element->find('.file a', 0)->href;
$item['imageThumb'] = $element->find('.file img', 0)->src;
if(!isset($item['imageThumb']) and strpos($item['image'], '.swf') !== false)
$item['imageThumb'] = 'http://i.imgur.com/eO0cxf9.jpg';
}
if(!empty($element->find('span.subject', 0)->innertext)) {
$item['subject'] = $element->find('span.subject', 0)->innertext;
}
$item['title'] = 'reply ' . $item['id'] . ' | ' . $item['author'];
if(isset($item['subject'])) {
$item['title'] = $item['subject'] . ' - ' . $item['title'];
}
$content = $element->find('.postMessage', 0)->innertext;
$content = str_replace('href="#p', 'href="' . $this->getURI() . '#p', $content);
$item['content'] = '<span id="' . $item['id'] . '">' . $content . '</span>';
if(isset($item['image'])) {
$item['content'] = '<a href="'
. $item['image']
. '"><img alt="'
. $item['id']
. '" src="'
. $item['imageThumb']
. '" /></a><br>'
.$item['content'];
}
$this->items[] = $item;
}
$this->items = array_reverse($this->items);
}
$this->items = array_reverse($this->items);
}
}

View File

@@ -1,166 +1,173 @@
<?php
class FuturaSciencesBridge extends FeedExpander {
const MAINTAINER = 'ORelio';
const NAME = 'Futura-Sciences Bridge';
const URI = 'http://www.futura-sciences.com/';
const DESCRIPTION = 'Returns the newest articles.';
const MAINTAINER = 'ORelio';
const NAME = 'Futura-Sciences Bridge';
const URI = 'http://www.futura-sciences.com/';
const DESCRIPTION = 'Returns the newest articles.';
const PARAMETERS = array( array(
'feed'=> array(
'name'=>'Feed',
'type'=>'list',
'values'=>array(
'Les flux multi-magazines'=>array(
'Les dernières actualités de Futura-Sciences'=>'actualites',
'Les dernières définitions de Futura-Sciences'=>'definitions',
'Les dernières photos de Futura-Sciences'=>'photos',
'Les dernières questions - réponses de Futura-Sciences'=>'questions-reponses',
'Les derniers dossiers de Futura-Sciences'=>'dossiers'
),
'Les flux Services'=> array(
'Les cartes virtuelles de Futura-Sciences'=>'services/cartes-virtuelles',
'Les fonds d\'écran de Futura-Sciences'=>'services/fonds-ecran'
),
'Les flux Santé'=>array(
'Les dernières actualités de Futura-Santé'=>'sante/actualites',
'Les dernières définitions de Futura-Santé'=>'sante/definitions',
'Les dernières questions-réponses de Futura-Santé'=>'sante/question-reponses',
'Les derniers dossiers de Futura-Santé'=>'sante/dossiers'
),
'Les flux High-Tech'=>array(
'Les dernières actualités de Futura-High-Tech'=>'high-tech/actualites',
'Les dernières astuces de Futura-High-Tech'=>'high-tech/question-reponses',
'Les dernières définitions de Futura-High-Tech'=>'high-tech/definitions',
'Les derniers dossiers de Futura-High-Tech'=>'high-tech/dossiers'
),
'Les flux Espace'=>array(
'Les dernières actualités de Futura-Espace'=>'espace/actualites',
'Les dernières définitions de Futura-Espace'=>'espace/definitions',
'Les dernières questions-réponses de Futura-Espace'=>'espace/question-reponses',
'Les derniers dossiers de Futura-Espace'=>'espace/dossiers'
),
'Les flux Environnement'=>array(
'Les dernières actualités de Futura-Environnement'=>'environnement/actualites',
'Les dernières définitions de Futura-Environnement'=>'environnement/definitions',
'Les dernières questions-réponses de Futura-Environnement'=>'environnement/question-reponses',
'Les derniers dossiers de Futura-Environnement'=>'environnement/dossiers'
),
'Les flux Maison'=>array(
'Les dernières actualités de Futura-Maison'=>'maison/actualites',
'Les dernières astuces de Futura-Maison'=>'maison/question-reponses',
'Les dernières définitions de Futura-Maison'=>'maison/definitions',
'Les derniers dossiers de Futura-Maison'=>'maison/dossiers'
),
'Les flux Nature'=>array(
'Les dernières actualités de Futura-Nature'=>'nature/actualites',
'Les dernières définitions de Futura-Nature'=>'nature/definitions',
'Les dernières questions-réponses de Futura-Nature'=>'nature/question-reponses',
'Les derniers dossiers de Futura-Nature'=>'nature/dossiers'
),
'Les flux Terre'=>array(
'Les dernières actualités de Futura-Terre'=>'terre/actualites',
'Les dernières définitions de Futura-Terre'=>'terre/definitions',
'Les dernières questions-réponses de Futura-Terre'=>'terre/question-reponses',
'Les derniers dossiers de Futura-Terre'=>'terre/dossiers'
),
'Les flux Matière'=>array(
'Les dernières actualités de Futura-Matière'=>'matiere/actualites',
'Les dernières définitions de Futura-Matière'=>'matiere/definitions',
'Les dernières questions-réponses de Futura-Matière'=>'matiere/question-reponses',
'Les derniers dossiers de Futura-Matière'=>'matiere/dossiers'
),
'Les flux Mathématiques'=>array(
'Les dernières actualités de Futura-Mathématiques'=>'mathematiques/actualites',
'Les derniers dossiers de Futura-Mathématiques'=>'mathematiques/dossiers'
)
)
)
));
const PARAMETERS = array( array(
'feed' => array(
'name' => 'Feed',
'type' => 'list',
'values' => array(
'Les flux multi-magazines' => array(
'Les dernières actualités de Futura-Sciences' => 'actualites',
'Les dernières définitions de Futura-Sciences' => 'definitions',
'Les dernières photos de Futura-Sciences' => 'photos',
'Les dernières questions - réponses de Futura-Sciences' => 'questions-reponses',
'Les derniers dossiers de Futura-Sciences' => 'dossiers'
),
'Les flux Services' => array(
'Les cartes virtuelles de Futura-Sciences' => 'services/cartes-virtuelles',
'Les fonds d\'écran de Futura-Sciences' => 'services/fonds-ecran'
),
'Les flux Santé' => array(
'Les dernières actualités de Futura-Santé' => 'sante/actualites',
'Les dernières définitions de Futura-Santé' => 'sante/definitions',
'Les dernières questions-réponses de Futura-Santé' => 'sante/question-reponses',
'Les derniers dossiers de Futura-Santé' => 'sante/dossiers'
),
'Les flux High-Tech' => array(
'Les dernières actualités de Futura-High-Tech' => 'high-tech/actualites',
'Les dernières astuces de Futura-High-Tech' => 'high-tech/question-reponses',
'Les dernières définitions de Futura-High-Tech' => 'high-tech/definitions',
'Les derniers dossiers de Futura-High-Tech' => 'high-tech/dossiers'
),
'Les flux Espace' => array(
'Les dernières actualités de Futura-Espace' => 'espace/actualites',
'Les dernières définitions de Futura-Espace' => 'espace/definitions',
'Les dernières questions-réponses de Futura-Espace' => 'espace/question-reponses',
'Les derniers dossiers de Futura-Espace' => 'espace/dossiers'
),
'Les flux Environnement' => array(
'Les dernières actualités de Futura-Environnement' => 'environnement/actualites',
'Les dernières définitions de Futura-Environnement' => 'environnement/definitions',
'Les dernières questions-réponses de Futura-Environnement' => 'environnement/question-reponses',
'Les derniers dossiers de Futura-Environnement' => 'environnement/dossiers'
),
'Les flux Maison' => array(
'Les dernières actualités de Futura-Maison' => 'maison/actualites',
'Les dernières astuces de Futura-Maison' => 'maison/question-reponses',
'Les dernières définitions de Futura-Maison' => 'maison/definitions',
'Les derniers dossiers de Futura-Maison' => 'maison/dossiers'
),
'Les flux Nature' => array(
'Les dernières actualités de Futura-Nature' => 'nature/actualites',
'Les dernières définitions de Futura-Nature' => 'nature/definitions',
'Les dernières questions-réponses de Futura-Nature' => 'nature/question-reponses',
'Les derniers dossiers de Futura-Nature' => 'nature/dossiers'
),
'Les flux Terre' => array(
'Les dernières actualités de Futura-Terre' => 'terre/actualites',
'Les dernières définitions de Futura-Terre' => 'terre/definitions',
'Les dernières questions-réponses de Futura-Terre' => 'terre/question-reponses',
'Les derniers dossiers de Futura-Terre' => 'terre/dossiers'
),
'Les flux Matière' => array(
'Les dernières actualités de Futura-Matière' => 'matiere/actualites',
'Les dernières définitions de Futura-Matière' => 'matiere/definitions',
'Les dernières questions-réponses de Futura-Matière' => 'matiere/question-reponses',
'Les derniers dossiers de Futura-Matière' => 'matiere/dossiers'
),
'Les flux Mathématiques' => array(
'Les dernières actualités de Futura-Mathématiques' => 'mathematiques/actualites',
'Les derniers dossiers de Futura-Mathématiques' => 'mathematiques/dossiers'
)
)
)
));
public function collectData(){
$url = self::URI . 'rss/' . $this->getInput('feed') . '.xml';
$this->collectExpandableDatas($url, 10);
}
public function collectData(){
$url = self::URI . 'rss/' . $this->getInput('feed') . '.xml';
$this->collectExpandableDatas($url, 10);
}
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
$item['uri'] = str_replace('#xtor=RSS-8', '', $item['uri']);
$article = getSimpleHTMLDOMCached($item['uri'])
or returnServerError('Could not request Futura-Sciences: ' . $item['uri']);
$item['content'] = $this->ExtractArticleContent($article);
$item['author'] = empty($this->ExtractAuthor($article)) ? $item['author'] : $this->ExtractAuthor($article);
return $item;
}
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
$item['uri'] = str_replace('#xtor=RSS-8', '', $item['uri']);
$article = getSimpleHTMLDOMCached($item['uri'])
or returnServerError('Could not request Futura-Sciences: ' . $item['uri']);
$item['content'] = $this->extractArticleContent($article);
$author = $this->extractAuthor($article);
$item['author'] = empty($author) ? $item['author'] : $author;
return $item;
}
function StripWithDelimiters($string, $start, $end) {
while (strpos($string, $start) !== false) {
$section_to_remove = substr($string, strpos($string, $start));
$section_to_remove = substr($section_to_remove, 0, strpos($section_to_remove, $end) + strlen($end));
$string = str_replace($section_to_remove, '', $string);
} return $string;
}
private function stripWithDelimiters($string, $start, $end){
while(strpos($string, $start) !== false) {
$section_to_remove = substr($string, strpos($string, $start));
$section_to_remove = substr($section_to_remove, 0, strpos($section_to_remove, $end) + strlen($end));
$string = str_replace($section_to_remove, '', $string);
} return $string;
}
function StripRecursiveHTMLSection($string, $tag_name, $tag_start) {
$open_tag = '<'.$tag_name;
$close_tag = '</'.$tag_name.'>';
$close_tag_length = strlen($close_tag);
if (strpos($tag_start, $open_tag) === 0) {
while (strpos($string, $tag_start) !== false) {
$max_recursion = 100;
$section_to_remove = null;
$section_start = strpos($string, $tag_start);
$search_offset = $section_start;
do {
$max_recursion--;
$section_end = strpos($string, $close_tag, $search_offset);
$search_offset = $section_end + $close_tag_length;
$section_to_remove = substr($string, $section_start, $section_end - $section_start + $close_tag_length);
$open_tag_count = substr_count($section_to_remove, $open_tag);
$close_tag_count = substr_count($section_to_remove, $close_tag);
} while ($open_tag_count > $close_tag_count && $max_recursion > 0);
$string = str_replace($section_to_remove, '', $string);
}
}
return $string;
}
private function stripRecursiveHTMLSection($string, $tag_name, $tag_start){
$open_tag = '<' . $tag_name;
$close_tag = '</' . $tag_name . '>';
$close_tag_length = strlen($close_tag);
if(strpos($tag_start, $open_tag) === 0) {
while(strpos($string, $tag_start) !== false) {
$max_recursion = 100;
$section_to_remove = null;
$section_start = strpos($string, $tag_start);
$search_offset = $section_start;
do {
$max_recursion--;
$section_end = strpos($string, $close_tag, $search_offset);
$search_offset = $section_end + $close_tag_length;
$section_to_remove = substr($string, $section_start, $section_end - $section_start + $close_tag_length);
$open_tag_count = substr_count($section_to_remove, $open_tag);
$close_tag_count = substr_count($section_to_remove, $close_tag);
} while ($open_tag_count > $close_tag_count && $max_recursion > 0);
$string = str_replace($section_to_remove, '', $string);
}
}
return $string;
}
function ExtractArticleContent($article){
$contents = $article->find('div.content', 0)->innertext;
private function extractArticleContent($article){
$contents = $article->find('section.article-text-classic', 0)->innertext;
$headline = trim($article->find('p.description', 0)->plaintext);
if(!empty($headline))
$headline = '<p><b>' . $headline . '</b></p>';
foreach (array(
'<div class="clear',
'<div class="sharebar2',
'<div class="diaporamafullscreen"',
'<div style="margin-bottom:10px;" class="noprint"',
'<div class="ficheprevnext',
'<div class="bar noprint',
'<div class="toolbar noprint',
'<div class="addthis_toolbox',
'<div class="noprint',
'<div class="bg bglight border border-full noprint',
'<div class="httplogbar-wrapper noprint',
'<div id="forumcomments'
) as $div_start) {
$contents = $this->StripRecursiveHTMLSection($contents , 'div', $div_start);
}
foreach (array(
'<div class="clear',
'<div class="sharebar2',
'<div class="diaporamafullscreen"',
'<div class="module social-button',
'<div style="margin-bottom:10px;" class="noprint"',
'<div class="ficheprevnext',
'<div class="bar noprint',
'<div class="toolbar noprint',
'<div class="addthis_toolbox',
'<div class="noprint',
'<div class="bg bglight border border-full noprint',
'<div class="httplogbar-wrapper noprint',
'<div id="forumcomments',
'<div ng-if="active"'
) as $div_start) {
$contents = $this->stripRecursiveHTMLSection($contents, 'div', $div_start);
}
$contents = $this->StripWithDelimiters($contents, '<hr ', '/>');
$contents = $this->StripWithDelimiters($contents, '<p class="content-date', '</p>');
$contents = $this->StripWithDelimiters($contents, '<h1 class="content-title', '</h1>');
$contents = $this->StripWithDelimiters($contents, 'fs:definition="', '"');
$contents = $this->StripWithDelimiters($contents, 'fs:xt:clicktype="', '"');
$contents = $this->StripWithDelimiters($contents, 'fs:xt:clickname="', '"');
$contents = $this->stripWithDelimiters($contents, '<hr ', '/>');
$contents = $this->stripWithDelimiters($contents, '<p class="content-date', '</p>');
$contents = $this->stripWithDelimiters($contents, '<h1 class="content-title', '</h1>');
$contents = $this->stripWithDelimiters($contents, 'fs:definition="', '"');
$contents = $this->stripWithDelimiters($contents, 'fs:xt:clicktype="', '"');
$contents = $this->stripWithDelimiters($contents, 'fs:xt:clickname="', '"');
$contents = $this->stripWithDelimiters($contents, '<script ', '</script>');
return $contents;
}
return $headline . trim($contents);
}
// Extracts the author from an article or element
function ExtractAuthor($article){
$article_author = $article->find('span.author', 0);
if($article_author){
return trim(str_replace(', Futura-Sciences', '', $article_author->plaintext));
}
return '';
}
// Extracts the author from an article or element
private function extractAuthor($article){
$article_author = $article->find('h3.epsilon', 0);
if($article_author) {
return trim(str_replace(', Futura-Sciences', '', $article_author->plaintext));
}
return '';
}
}

View File

@@ -1,124 +1,157 @@
<?php
class GBAtempBridge extends BridgeAbstract {
const MAINTAINER = 'ORelio';
const NAME = 'GBAtemp';
const URI = 'http://gbatemp.net/';
const DESCRIPTION = 'GBAtemp is a user friendly underground video game community.';
const MAINTAINER = 'ORelio';
const NAME = 'GBAtemp';
const URI = 'https://gbatemp.net/';
const DESCRIPTION = 'GBAtemp is a user friendly underground video game community.';
const PARAMETERS = array( array(
'type'=>array(
'name'=>'Type',
'type'=>'list',
'required'=>true,
'values'=>array(
'News'=>'N',
'Reviews'=>'R',
'Tutorials'=>'T',
'Forum'=>'F'
)
)
));
const PARAMETERS = array( array(
'type' => array(
'name' => 'Type',
'type' => 'list',
'required' => true,
'values' => array(
'News' => 'N',
'Reviews' => 'R',
'Tutorials' => 'T',
'Forum' => 'F'
)
)
));
private function ExtractFromDelimiters($string, $start, $end) {
if (strpos($string, $start) !== false) {
$section_retrieved = substr($string, strpos($string, $start) + strlen($start));
$section_retrieved = substr($section_retrieved, 0, strpos($section_retrieved, $end));
return $section_retrieved;
} return false;
}
private function extractFromDelimiters($string, $start, $end){
if(strpos($string, $start) !== false) {
$section_retrieved = substr($string, strpos($string, $start) + strlen($start));
$section_retrieved = substr($section_retrieved, 0, strpos($section_retrieved, $end));
return $section_retrieved;
}
private function StripWithDelimiters($string, $start, $end) {
while (strpos($string, $start) !== false) {
$section_to_remove = substr($string, strpos($string, $start));
$section_to_remove = substr($section_to_remove, 0, strpos($section_to_remove, $end) + strlen($end));
$string = str_replace($section_to_remove, '', $string);
} return $string;
}
return false;
}
private function build_item($uri, $title, $author, $timestamp, $content) {
$item = array();
$item['uri'] = $uri;
$item['title'] = $title;
$item['author'] = $author;
$item['timestamp'] = $timestamp;
$item['content'] = $content;
return $item;
}
private function stripWithDelimiters($string, $start, $end){
while(strpos($string, $start) !== false) {
$section_to_remove = substr($string, strpos($string, $start));
$section_to_remove = substr($section_to_remove, 0, strpos($section_to_remove, $end) + strlen($end));
$string = str_replace($section_to_remove, '', $string);
}
private function cleanup_post_content($content, $site_url) {
$content = str_replace(':arrow:', '&#x27a4;', $content);
$content = str_replace('href="attachments/', 'href="'.$site_url.'attachments/', $content);
$content = $this->StripWithDelimiters($content, '<script', '</script>');
return $content;
}
return $string;
}
private function fetch_post_content($uri, $site_url) {
$html = getSimpleHTMLDOM($uri);
if(!$html){
return 'Could not request GBAtemp '.$uri;
}
private function buildItem($uri, $title, $author, $timestamp, $content){
$item = array();
$item['uri'] = $uri;
$item['title'] = $title;
$item['author'] = $author;
$item['timestamp'] = $timestamp;
$item['content'] = $content;
return $item;
}
$content = $html->find('div.messageContent', 0)->innertext;
return $this->cleanup_post_content($content, $site_url);
}
private function cleanupPostContent($content, $site_url){
$content = str_replace(':arrow:', '&#x27a4;', $content);
$content = str_replace('href="attachments/', 'href="'.$site_url.'attachments/', $content);
$content = $this->stripWithDelimiters($content, '<script', '</script>');
return $content;
}
public function collectData(){
private function fetchPostContent($uri, $site_url){
$html = getSimpleHTMLDOM($uri);
if(!$html) {
return 'Could not request GBAtemp ' . $uri;
}
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not request GBAtemp.');
$content = $html->find('div.messageContent', 0)->innertext;
return $this->cleanupPostContent($content, $site_url);
}
switch($this->getInput('type')){
case 'N':
foreach ($html->find('li[class=news_item full]') as $newsItem) {
$url = self::URI.$newsItem->find('a', 0)->href;
$time = intval($this->ExtractFromDelimiters($newsItem->find('abbr.DateTime', 0)->outertext, 'data-time="', '"'));
$author = $newsItem->find('a.username', 0)->plaintext;
$title = $newsItem->find('a', 1)->plaintext;
$content = $this->fetch_post_content($url, self::URI);
$this->items[] = $this->build_item($url, $title, $author, $time, $content);
}
case 'R':
foreach ($html->find('li.portal_review') as $reviewItem) {
$url = self::URI.$reviewItem->find('a', 0)->href;
$title = $reviewItem->find('span.review_title', 0)->plaintext;
$content = getSimpleHTMLDOM($url) or returnServerError('Could not request GBAtemp: '.$uri);
$author = $content->find('a.username', 0)->plaintext;
$time = intval($this->ExtractFromDelimiters($content->find('abbr.DateTime', 0)->outertext, 'data-time="', '"'));
$intro = '<p><b>'.($content->find('div#review_intro', 0)->plaintext).'</b></p>';
$review = $content->find('div#review_main', 0)->innertext;
$subheader = '<p><b>'.$content->find('div.review_subheader', 0)->plaintext.'</b></p>';
$procons = $content->find('table.review_procons', 0)->outertext;
$scores = $content->find('table.reviewscores', 0)->outertext;
$content = $this->cleanup_post_content($intro.$review.$subheader.$procons.$scores, self::URI);
$this->items[] = $this->build_item($url, $title, $author, $time, $content);
}
case 'T':
foreach ($html->find('li.portal-tutorial') as $tutorialItem) {
$url = self::URI.$tutorialItem->find('a', 0)->href;
$title = $tutorialItem->find('a', 0)->plaintext;
$time = intval($this->ExtractFromDelimiters($tutorialItem->find('abbr.DateTime', 0)->outertext, 'data-time="', '"'));
$author = $tutorialItem->find('a.username', 0)->plaintext;
$content = $this->fetch_post_content($url, self::URI);
$this->items[] = $this->build_item($url, $title, $author, $time, $content);
}
case 'F':
foreach ($html->find('li.rc_item') as $postItem) {
$url = self::URI.$postItem->find('a', 1)->href;
$title = $postItem->find('a', 1)->plaintext;
$time = intval($this->ExtractFromDelimiters($postItem->find('abbr.DateTime', 0)->outertext, 'data-time="', '"'));
$author = $postItem->find('a.username', 0)->plaintext;
$content = $this->fetch_post_content($url, self::URI);
$this->items[] = $this->build_item($url, $title, $author, $time, $content);
}
}
}
public function collectData(){
public function getName() {
$type=array_search(
$this->getInput('type'),
self::PARAMETERS[$this->queriedContext]['type']['values']
);
return 'GBAtemp '.$type.' Bridge';
}
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not request GBAtemp.');
switch($this->getInput('type')) {
case 'N':
foreach($html->find('li[class=news_item full]') as $newsItem) {
$url = self::URI . $newsItem->find('a', 0)->href;
$time = intval(
$this->extractFromDelimiters(
$newsItem->find('abbr.DateTime', 0)->outertext,
'data-time="',
'"'
)
);
$author = $newsItem->find('a.username', 0)->plaintext;
$title = $newsItem->find('a', 1)->plaintext;
$content = $this->fetchPostContent($url, self::URI);
$this->items[] = $this->buildItem($url, $title, $author, $time, $content);
}
case 'R':
foreach($html->find('li.portal_review') as $reviewItem) {
$url = self::URI . $reviewItem->find('a', 0)->href;
$title = $reviewItem->find('span.review_title', 0)->plaintext;
$content = getSimpleHTMLDOM($url)
or returnServerError('Could not request GBAtemp: ' . $uri);
$author = $content->find('a.username', 0)->plaintext;
$time = intval(
$this->extractFromDelimiters(
$content->find('abbr.DateTime', 0)->outertext,
'data-time="',
'"'
)
);
$intro = '<p><b>' . ($content->find('div#review_intro', 0)->plaintext) . '</b></p>';
$review = $content->find('div#review_main', 0)->innertext;
$subheader = '<p><b>' . $content->find('div.review_subheader', 0)->plaintext . '</b></p>';
$procons = $content->find('table.review_procons', 0)->outertext;
$scores = $content->find('table.reviewscores', 0)->outertext;
$content = $this->cleanupPostContent($intro . $review . $subheader . $procons . $scores, self::URI);
$this->items[] = $this->buildItem($url, $title, $author, $time, $content);
}
case 'T':
foreach($html->find('li.portal-tutorial') as $tutorialItem) {
$url = self::URI . $tutorialItem->find('a', 0)->href;
$title = $tutorialItem->find('a', 0)->plaintext;
$time = intval(
$this->extractFromDelimiters(
$tutorialItem->find('abbr.DateTime', 0)->outertext,
'data-time="',
'"'
)
);
$author = $tutorialItem->find('a.username', 0)->plaintext;
$content = $this->fetchPostContent($url, self::URI);
$this->items[] = $this->buildItem($url, $title, $author, $time, $content);
}
case 'F':
foreach($html->find('li.rc_item') as $postItem) {
$url = self::URI . $postItem->find('a', 1)->href;
$title = $postItem->find('a', 1)->plaintext;
$time = intval(
$this->extractFromDelimiters(
$postItem->find('abbr.DateTime', 0)->outertext,
'data-time="',
'"'
)
);
$author = $postItem->find('a.username', 0)->plaintext;
$content = $this->fetchPostContent($url, self::URI);
$this->items[] = $this->buildItem($url, $title, $author, $time, $content);
}
}
}
public function getName() {
if(!is_null($this->getInput('type'))) {
$type = array_search(
$this->getInput('type'),
self::PARAMETERS[$this->queriedContext]['type']['values']
);
return 'GBAtemp ' . $type . ' Bridge';
}
return parent::getName();
}
}

View File

@@ -1,21 +1,35 @@
<?php
require_once('DanbooruBridge.php');
class GelbooruBridge extends DanbooruBridge{
class GelbooruBridge extends DanbooruBridge {
const MAINTAINER = "mitsukarenai";
const NAME = "Gelbooru";
const URI = "http://gelbooru.com/";
const DESCRIPTION = "Returns images from given page";
const MAINTAINER = 'mitsukarenai';
const NAME = 'Gelbooru';
const URI = 'http://gelbooru.com/';
const DESCRIPTION = 'Returns images from given page';
const PATHTODATA='.thumb';
const IDATTRIBUTE='id';
const PATHTODATA = '.thumb';
const IDATTRIBUTE = 'id';
const TAGATTRIBUTE = 'title';
const PIDBYPAGE=63;
const PIDBYPAGE = 63;
protected function getFullURI(){
return $this->getURI().'index.php?page=post&s=list&'
.'&pid='.($this->getInput('p')?($this->getInput('p') -1)*static::PIDBYPAGE:'')
.'&tags='.urlencode($this->getInput('t'));
}
protected function getFullURI(){
return $this->getURI()
. 'index.php?page=post&s=list&pid='
. ($this->getInput('p') ? ($this->getInput('p') - 1) * static::PIDBYPAGE : '')
. '&tags=' . urlencode($this->getInput('t'));
}
protected function getTags($element){
$tags = parent::getTags($element);
$tags = explode(' ', $tags);
// Remove statistics from the tags list (identified by colon)
foreach($tags as $key => $tag) {
if(strpos($tag, ':') !== false) unset($tags[$key]);
}
return implode(' ', $tags);
}
}

View File

@@ -1,71 +1,76 @@
<?php
define('GIPHY_LIMIT', 10);
class GiphyBridge extends BridgeAbstract{
class GiphyBridge extends BridgeAbstract {
const MAINTAINER = "kraoc";
const NAME = "Giphy Bridge";
const URI = "http://giphy.com/";
const MAINTAINER = 'kraoc';
const NAME = 'Giphy Bridge';
const URI = 'http://giphy.com/';
const CACHE_TIMEOUT = 300; //5min
const DESCRIPTION = "Bridge for giphy.com";
const DESCRIPTION = 'Bridge for giphy.com';
const PARAMETERS = array( array(
's'=>array(
'name'=>'search tag',
'required'=>true
),
'n'=>array(
'name'=>'max number of returned items',
'type'=>'number'
)
));
const PARAMETERS = array( array(
's' => array(
'name' => 'search tag',
'required' => true
),
'n' => array(
'name' => 'max number of returned items',
'type' => 'number'
)
));
public function collectData(){
$html = '';
$base_url = 'http://giphy.com';
$html = getSimpleHTMLDOM(self::URI.'/search/'.urlencode($this->getInput('s').'/'))
or returnServerError('No results for this query.');
$base_url = 'http://giphy.com';
$html = getSimpleHTMLDOM(self::URI . '/search/' . urlencode($this->getInput('s') . '/'))
or returnServerError('No results for this query.');
$max = GIPHY_LIMIT;
if ($this->getInput('n')) {
$max = $this->getInput('n');
}
$max = GIPHY_LIMIT;
if($this->getInput('n')) {
$max = $this->getInput('n');
}
$limit = 0;
$kw = urlencode($this->getInput('s'));
foreach($html->find('div.hoverable-gif') as $entry) {
if($limit < $max) {
$node = $entry->first_child();
$href = $node->getAttribute('href');
$limit = 0;
$kw = urlencode($this->getInput('s'));
foreach($html->find('div.hoverable-gif') as $entry) {
if($limit < $max) {
$node = $entry->first_child();
$href = $node->getAttribute('href');
$html2 = getSimpleHTMLDOM(self::URI . $href)
or returnServerError('No results for this query.');
$figure = $html2->getElementByTagName('figure');
$img = $figure->firstChild();
$caption = $figure->lastChild();
$html2 = getSimpleHTMLDOM(self::URI . $href)
or returnServerError('No results for this query.');
$figure = $html2->getElementByTagName('figure');
$img = $figure->firstChild();
$caption = $figure->lastChild();
$item = array();
$item['id'] = $img->getAttribute('data-gif_id');
$item['uri'] = $img->getAttribute('data-bitly_gif_url');
$item['username'] = 'Giphy - '.ucfirst($kw);
$title = $caption->innertext();
$title = preg_replace('/\s+/', ' ',$title);
$title = str_replace('animated GIF', '', $title);
$title = str_replace($kw, '', $title);
$title = preg_replace('/\s+/', ' ',$title);
$title = trim($title);
if (strlen($title) <= 0) {
$title = $item['id'];
}
$item['title'] = trim($title);
$item['content'] =
'<a href="'.$item['uri'].'">'
.'<img src="'.$img->getAttribute('src').'" width="'.$img->getAttribute('data-original-width').'" height="'.$img->getAttribute('data-original-height').'" />'
.'</a>';
$item = array();
$item['id'] = $img->getAttribute('data-gif_id');
$item['uri'] = $img->getAttribute('data-bitly_gif_url');
$item['username'] = 'Giphy - ' . ucfirst($kw);
$title = $caption->innertext();
$title = preg_replace('/\s+/', ' ', $title);
$title = str_replace('animated GIF', '', $title);
$title = str_replace($kw, '', $title);
$title = preg_replace('/\s+/', ' ', $title);
$title = trim($title);
if(strlen($title) <= 0) {
$title = $item['id'];
}
$item['title'] = trim($title);
$item['content'] = '<a href="'
. $item['uri']
. '"><img src="'
. $img->getAttribute('src')
. '" width="'
. $img->getAttribute('data-original-width')
. '" height="'
. $img->getAttribute('data-original-height')
. '" /></a>';
$this->items[] = $item;
$limit++;
}
}
$this->items[] = $item;
$limit++;
}
}
}
}

View File

@@ -1,190 +1,192 @@
<?php
class GithubIssueBridge extends BridgeAbstract{
class GithubIssueBridge extends BridgeAbstract {
const MAINTAINER = 'Pierre Mazière';
const NAME = 'Github Issue';
const URI = 'https://github.com/';
const CACHE_TIMEOUT = 600; // 10min
const DESCRIPTION = 'Returns the issues or comments of an issue of a github project';
const MAINTAINER = 'Pierre Mazière';
const NAME = 'Github Issue';
const URI = 'https://github.com/';
const CACHE_TIMEOUT = 600; // 10min
const DESCRIPTION = 'Returns the issues or comments of an issue of a github project';
const PARAMETERS=array(
'global'=>array (
'u'=>array(
'name'=>'User name',
'required'=>true
),
'p'=>array(
'name'=>'Project name',
'required'=>true
)
),
const PARAMETERS = array(
'global' => array(
'u' => array(
'name' => 'User name',
'required' => true
),
'p' => array(
'name' => 'Project name',
'required' => true
)
),
'Project Issues' => array(
'c' => array(
'name' => 'Show Issues Comments',
'type' => 'checkbox'
)
),
'Issue comments' => array(
'i' => array(
'name' => 'Issue number',
'type' => 'number',
'required' => 'true'
)
)
);
'Project Issues'=>array(
'c'=>array(
'name'=>'Show Issues Comments',
'type'=>'checkbox'
)
),
'Issue comments'=>array(
'i'=>array(
'name'=>'Issue number',
'type'=>'number',
'required'=>'true'
)
)
);
public function getName(){
$name = $this->getInput('u') . '/' . $this->getInput('p');
switch($this->queriedContext) {
case 'Project Issues':
if($this->getInput('c')) {
$prefix = static::NAME . 's comments for ';
} else {
$prefix = static::NAME . 's for ';
}
$name = $prefix . $name;
break;
case 'Issue comments':
$name = static::NAME . ' ' . $name . ' #' . $this->getInput('i');
break;
default: return parent::getName();
}
return $name;
}
public function getName(){
$name=$this->getInput('u').'/'.$this->getInput('p');
switch($this->queriedContext){
case 'Project Issues':
if($this->getInput('c')){
$prefix=static::NAME.'s comments for ';
}else{
$prefix=static::NAME.'s for ';
}
$name=$prefix.$name;
break;
case 'Issue comments':
$name=static::NAME.' '.$name.' #'.$this->getInput('i');
break;
}
return $name;
}
public function getURI(){
if(!is_null($this->getInput('u')) && !is_null($this->getInput('p'))) {
$uri = static::URI . $this->getInput('u') . '/' . $this->getInput('p') . '/issues';
if($this->queriedContext === 'Issue comments') {
$uri .= '/' . $this->getInput('i');
} elseif($this->getInput('c')) {
$uri .= '?q=is%3Aissue+sort%3Aupdated-desc';
}
return $uri;
}
public function getURI(){
$uri = static::URI.$this->getInput('u').'/'.$this->getInput('p').'/issues';
if($this->queriedContext==='Issue comments'){
$uri.='/'.$this->getInput('i');
}else if($this->getInput('c')){
$uri.='?q=is%3Aissue+sort%3Aupdated-desc';
}
return $uri;
}
return parent::getURI();
}
protected function extractIssueComment($issueNbr,$title,$comment){
$class=$comment->getAttribute('class');
$classes=explode(' ',$class);
$event=false;
if(in_array('discussion-item',$classes)){
$event=true;
}
protected function extractIssueComment($issueNbr, $title, $comment){
$class = $comment->getAttribute('class');
$classes = explode(' ', $class);
$event = false;
if(in_array('discussion-item', $classes)) {
$event = true;
}
$author='unknown';
if($comment->find('.author',0)){
$author=$comment->find('.author',0)->plaintext;
}
$author = 'unknown';
if($comment->find('.author', 0)) {
$author = $comment->find('.author', 0)->plaintext;
}
$uri=static::URI.$this->getInput('u').'/'.$this->getInput('p').'/issues/'
.$issueNbr;
$uri = static::URI . $this->getInput('u') . '/' . $this->getInput('p') . '/issues/' . $issueNbr;
$comment=$comment->firstChild();
if(!$event){
$comment=$comment->nextSibling();
}
$comment = $comment->firstChild();
if(!$event) {
$comment = $comment->nextSibling();
}
if($event){
$title.=' / '.substr($class,strpos($class,'discussion-item-')+strlen('discussion-item-'));
if(!$comment->hasAttribute('id')){
$items=array();
$timestamp=strtotime($comment->find('relative-time',0)->getAttribute('datetime'));
$content=$comment->innertext;
while($comment=$comment->nextSibling()){
$item=array();
$item['author']=$author;
$item['title']=html_entity_decode($title,ENT_QUOTES,'UTF-8');
$item['timestamp']=$timestamp;
$item['content']=$content.'<p>'.$comment->children(1)->innertext.'</p>';
$item['uri']=$uri.'#'.$comment->children(1)->getAttribute('id');
$items[]=$item;
}
return $items;
}
$content=$comment->parent()->innertext;
}else{
$title.=' / '.trim($comment->firstChild()->plaintext);
$content="<pre>".$comment->find('.comment-body',0)->innertext."</pre>";
}
if($event) {
$title .= ' / ' . substr($class, strpos($class, 'discussion-item-') + strlen('discussion-item-'));
if(!$comment->hasAttribute('id')) {
$items = array();
$timestamp = strtotime($comment->find('relative-time', 0)->getAttribute('datetime'));
$content = $comment->innertext;
while($comment = $comment->nextSibling()) {
$item = array();
$item['author'] = $author;
$item['title'] = html_entity_decode($title, ENT_QUOTES, 'UTF-8');
$item['timestamp'] = $timestamp;
$item['content'] = $content . '<p>' . $comment->children(1)->innertext . '</p>';
$item['uri'] = $uri . '#' . $comment->children(1)->getAttribute('id');
$items[] = $item;
}
return $items;
}
$content = $comment->parent()->innertext;
} else {
$title .= ' / ' . trim($comment->firstChild()->plaintext);
$content = '<pre>' . $comment->find('.comment-body', 0)->innertext . '</pre>';
}
$item = array();
$item['author']=$author;
$item['uri']= $uri.'#'.$comment->getAttribute('id');
$item['title']=html_entity_decode($title,ENT_QUOTES,'UTF-8');
$item['timestamp']=strtotime($comment->find('relative-time',0)->getAttribute('datetime'));
$item['content']=$content;
return $item;
}
$item = array();
$item['author'] = $author;
$item['uri'] = $uri . '#' . $comment->getAttribute('id');
$item['title'] = html_entity_decode($title, ENT_QUOTES, 'UTF-8');
$item['timestamp'] = strtotime($comment->find('relative-time', 0)->getAttribute('datetime'));
$item['content'] = $content;
return $item;
}
protected function extractIssueComments($issue){
$items=array();
$title=$issue->find('.gh-header-title',0)->plaintext;
$issueNbr=trim(substr($issue->find('.gh-header-number',0)->plaintext,1));
$comments=$issue->find('.js-discussion',0);
foreach($comments->children() as $comment){
$classes=explode(' ',$comment->getAttribute('class'));
if(in_array('discussion-item',$classes) ||
in_array('timeline-comment-wrapper',$classes)
){
$item=$this->extractIssueComment($issueNbr,$title,$comment);
if(array_keys($item)!==range(0,count($item)-1)){
$item=array($item);
}
$items=array_merge($items,$item);
}
}
return $items;
}
protected function extractIssueComments($issue){
$items = array();
$title = $issue->find('.gh-header-title', 0)->plaintext;
$issueNbr = trim(substr($issue->find('.gh-header-number', 0)->plaintext, 1));
$comments = $issue->find('.js-discussion', 0);
foreach($comments->children() as $comment) {
$classes = explode(' ', $comment->getAttribute('class'));
if(in_array('discussion-item', $classes)
|| in_array('timeline-comment-wrapper', $classes)) {
$item = $this->extractIssueComment($issueNbr, $title, $comment);
if(array_keys($item) !== range(0, count($item) - 1)) {
$item = array($item);
}
$items = array_merge($items, $item);
}
}
return $items;
}
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('No results for Github Issue '.$this->getURI());
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('No results for Github Issue ' . $this->getURI());
switch($this->queriedContext){
case 'Issue comments':
$this->items=$this->extractIssueComments($html);
break;
case 'Project Issues':
foreach($html->find('.js-active-navigation-container .js-navigation-item') as $issue){
$info=$issue->find('.opened-by',0);
$issueNbr=substr(trim($info->plaintext),1,strpos(trim($info->plaintext),' '));
switch($this->queriedContext) {
case 'Issue comments':
$this->items = $this->extractIssueComments($html);
break;
case 'Project Issues':
foreach($html->find('.js-active-navigation-container .js-navigation-item') as $issue) {
$info = $issue->find('.opened-by', 0);
$issueNbr = substr(trim($info->plaintext), 1, strpos(trim($info->plaintext), ' '));
$item=array();
$item['content']='';
$item = array();
$item['content'] = '';
if($this->getInput('c')){
$uri=static::URI.$this->getInput('u').'/'.$this->getInput('p').'/issues/'.$issueNbr;
$issue=getSimpleHTMLDOMCached($uri,static::CACHE_TIMEOUT);
if($issue){
$this->items=array_merge($this->items,$this->extractIssueComments($issue));
continue;
}
$item['content']='Can not extract comments from '.$uri;
}
if($this->getInput('c')) {
$uri = static::URI . $this->getInput('u') . '/' . $this->getInput('p') . '/issues/' . $issueNbr;
$issue = getSimpleHTMLDOMCached($uri, static::CACHE_TIMEOUT);
if($issue) {
$this->items = array_merge($this->items, $this->extractIssueComments($issue));
continue;
}
$item['content'] = 'Can not extract comments from ' . $uri;
}
$item['author']=$info->find('a',0)->plaintext;
$item['timestamp']=strtotime($info->find('relative-time',0)->getAttribute('datetime'));
$item['title']=html_entity_decode(
$issue->find('.js-navigation-open',0)->plaintext,
ENT_QUOTES,
'UTF-8'
);
$comments=$issue->find('.col-5',0)->plaintext;
$item['content'].="\n".'Comments: '.($comments?$comments:'0');
$item['uri']=self::URI.$issue->find('.js-navigation-open',0)->getAttribute('href');
$this->items[]=$item;
}
break;
}
$item['author'] = $info->find('a', 0)->plaintext;
$item['timestamp'] = strtotime($info->find('relative-time', 0)->getAttribute('datetime'));
$item['title'] = html_entity_decode(
$issue->find('.js-navigation-open', 0)->plaintext,
ENT_QUOTES,
'UTF-8'
);
$comments = $issue->find('.col-5', 0)->plaintext;
$item['content'] .= "\n" . 'Comments: ' . ($comments ? $comments : '0');
$item['uri'] = self::URI . $issue->find('.js-navigation-open', 0)->getAttribute('href');
$this->items[] = $item;
}
break;
}
array_walk($this->items, function(&$item){
$item['content']=preg_replace('/\s+/',' ',$item['content']);
$item['content']=str_replace('href="/','href="'.static::URI,$item['content']);
$item['content']=str_replace(
'href="#',
'href="'.substr($item['uri'],0,strpos($item['uri'],'#')+1),
$item['content']
);
$item['title']=preg_replace('/\s+/',' ',$item['title']);
});
}
array_walk($this->items, function(&$item){
$item['content'] = preg_replace('/\s+/', ' ', $item['content']);
$item['content'] = str_replace('href="/', 'href="' . static::URI, $item['content']);
$item['content'] = str_replace(
'href="#',
'href="' . substr($item['uri'], 0, strpos($item['uri'], '#') + 1),
$item['content']
);
$item['title'] = preg_replace('/\s+/', ' ', $item['title']);
});
}
}

View File

@@ -0,0 +1,50 @@
<?php
class GithubSearchBridge extends BridgeAbstract {
const MAINTAINER = 'corenting';
const NAME = 'Github Repositories Search';
const URI = 'https://github.com/';
const CACHE_TIMEOUT = 600; // 10min
const DESCRIPTION = 'Returns a specified repositories search (sorted by recently updated)';
const PARAMETERS = array( array(
's' => array(
'type' => 'text',
'name' => 'Search query'
)
));
public function collectData(){
$params = array('utf8' => '✓',
'q' => urlencode($this->getInput('s')),
's' => 'updated',
'o' => 'desc',
'type' => 'Repositories');
$url = self::URI . 'search?' . http_build_query($params);
$html = getSimpleHTMLDOM($url)
or returnServerError('Error while downloading the website content');
foreach($html->find('div.repo-list-item') as $element) {
$item = array();
$uri = $element->find('h3 a', 0)->href;
$uri = substr(self::URI, 0, -1) . $uri;
$item['uri'] = $uri;
$title = $element->find('h3', 0)->plaintext;
$item['title'] = $title;
if (count($element->find('p')) == 2) {
$content = $element->find('p', 0)->innertext;
} else{
$content = '';
}
$item['content'] = $content;
$date = $element->find('relative-time', 0)->datetime;
$item['timestamp'] = strtotime($date);
$this->items[] = $item;
}
}
}

View File

@@ -1,22 +1,26 @@
<?php
class GizmodoBridge extends FeedExpander {
const MAINTAINER = "polopollo";
const NAME = "Gizmodo";
const URI = "http://gizmodo.com/";
const MAINTAINER = 'polopollo';
const NAME = 'Gizmodo';
const URI = 'http://gizmodo.com/';
const CACHE_TIMEOUT = 1800; // 30min
const DESCRIPTION = "Returns the newest posts from Gizmodo (full text).";
const DESCRIPTION = 'Returns the newest posts from Gizmodo (full text).';
protected function parseItem($item){
$item = parent::parseItem($item);
$articleHTMLContent = getSimpleHTMLDOMCached($item['uri']);
if(!$articleHTMLContent){
$text = 'Could not load '.$item['uri'];
}else{
if(!$articleHTMLContent) {
$text = 'Could not load ' . $item['uri'];
} else {
$text = $articleHTMLContent->find('div.entry-content', 0)->innertext;
foreach($articleHTMLContent->find('pagespeed_iframe') as $element) {
$text .= '<p>link to a iframe (could be a video): <a href="'.$element->src.'">'.$element->src.'</a></p><br>';
$text .= '<p>link to a iframe (could be a video): <a href="'
. $element->src
. '">'
. $element->src
. '</a></p><br>';
}
$text = strip_tags($text, '<p><b><a><blockquote><img><em>');

View File

@@ -0,0 +1,61 @@
<?php
class GoComicsBridge extends BridgeAbstract {
const MAINTAINER = 'sky';
const NAME = 'GoComics Unofficial RSS';
const URI = 'https://www.gocomics.com/';
const CACHE_TIMEOUT = 21600; // 6h
const DESCRIPTION = 'The Unofficial GoComics RSS';
const PARAMETERS = array( array(
'comicname' => array(
'name' => 'comicname',
'type' => 'text',
'required' => true
)
));
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request GoComics: ' . $this->getURI());
//Get info from first page
$author = preg_replace('/By /', '', $html->find('.media-subheading', 0)->plaintext);
$link = self::URI . $html->find('.gc-deck--cta-0', 0)->find('a', 0)->href;
for($i = 0; $i < 5; $i++) {
$item = array();
$page = getSimpleHTMLDOM($link)
or returnServerError('Could not request GoComics: ' . $link);
$imagelink = $page->find('.img-fluid', 1)->src;
$date = explode('/', $link);
$item['id'] = $imagelink;
$item['uri'] = $link;
$item['author'] = $author;
$item['title'] = 'GoComics ' . $this->getInput('comicname');
$item['timestamp'] = DateTime::createFromFormat('Ymd', $date[5] . $date[6] . $date[7])->getTimestamp();
$item['content'] = '<img src="' . $imagelink . '" />';
$link = self::URI . $page->find('.js-previous-comic', 0)->href;
$this->items[] = $item;
}
}
public function getURI(){
if(!is_null($this->getInput('comicname'))) {
return self::URI . urlencode($this->getInput('comicname'));
}
return parent::getURI();
}
public function getName(){
if(!is_null($this->getInput('comicname'))) {
return $this->getInput('comicname') . ' - GoComics';
}
return parent::getName();
}
}

View File

@@ -1,113 +1,111 @@
<?php
class GooglePlusPostBridge extends BridgeAbstract
{
class GooglePlusPostBridge extends BridgeAbstract{
protected $_title;
protected $_url;
const MAINTAINER = "Grummfy";
const NAME = "Google Plus Post Bridge";
const URI = "https://plus.google.com/";
const MAINTAINER = 'Grummfy';
const NAME = 'Google Plus Post Bridge';
const URI = 'https://plus.google.com/';
const CACHE_TIMEOUT = 600; //10min
const DESCRIPTION = "Returns user public post (without API).";
const DESCRIPTION = 'Returns user public post (without API).';
const PARAMETERS = array( array(
'username'=>array(
'name'=>'username or Id',
'required'=>true
)
));
const PARAMETERS = array( array(
'username' => array(
'name' => 'username or Id',
'required' => true
)
));
public function collectData(){
$username = $this->getInput('username');
// Usernames start with a + if it's not an ID
if(!is_numeric($username) && substr($username, 0, 1) !== '+') {
$username = '+' . $username;
}
public function collectData()
{
// get content parsed
// $html = getSimpleHTMLDOM(__DIR__ . '/../posts2.html'
$html = getSimpleHTMLDOM(self::URI . urlencode($this->getInput('username')) . '/posts'
// force language
, false, stream_context_create(array('http'=> array(
'header' => 'Accept-Language: fr,fr-be,fr-fr;q=0.8,en;q=0.4,en-us;q=0.2;*' . "\r\n"
)))
) OR returnServerError('No results for this query.');
$html = getSimpleHTMLDOMCached(self::URI . urlencode($username) . '/posts')
or returnServerError('No results for this query.');
// get title, url, ... there is a lot of intresting stuff in meta
$this->_title = $html->find('meta[property]', 0)->getAttribute('content');
$this->_url = $html->find('meta[itemprop=url]', 0)->getAttribute('content');
$this->_title = $html->find('meta[property=og:title]', 0)->getAttribute('content');
$this->_url = $html->find('meta[property=og:url]', 0)->getAttribute('content');
// foreach ($html->find('meta') as $e)
// {
// $item = array();
// $item['content'] = var_export($e->attr, true);
// $this->items[] = $item;
// }
// div[jsmodel=XNmfOc]
foreach($html->find('div.yt') as $post)
{
// I don't even know where to start with this discusting html...
foreach($html->find('div[jsname=WsjYwc]') as $post) {
$item = array();
// $item['content'] = $post->find('div.Al', 0)->innertext;
$item['username'] = $item['fullname'] = $post->find('header.lea h3 a', 0)->innertext;
$item['id'] = $post->getAttribute('id');
// $item['title'] = $item['fullname'] = $post->find('header.lea', 0)->plaintext;
$item['avatar'] = $post->find('div.ys img', 0)->src;
// var_dump((($post->find('a.o-U-s', 0)->getAllAttributes())));
$item['uri'] = self::URI . $post->find('a.o-U-s', 0)->href;
$item['timestamp'] = strtotime($post->find('a.o-U-s', 0)->plaintext);
$this->items[] = $item;
$item['author'] = $item['fullname'] = $post->find('div div div div a', 0)->innertext;
$item['id'] = $post->find('div div div', 0)->getAttribute('id');
$item['avatar'] = $post->find('div img', 0)->src;
$item['uri'] = self::URI . $post->find('div div div a', 1)->href;
$timestamp = $post->find('a.qXj2He span', 0);
if($timestamp) {
$item['timestamp'] = strtotime('+' . preg_replace(
'/[^0-9A-Za-z]/',
'',
$timestamp->getAttribute('aria-label')));
}
// hashtag to treat : https://plus.google.com/explore/tag
$hashtags = array();
foreach($post->find('a.d-s') as $hashtag)
{
$hashtags[ trim($hashtag->plaintext) ] = self::URI . $hashtag->href;
}
// $hashtags = array();
// foreach($post->find('a.d-s') as $hashtag){
// $hashtags[trim($hashtag->plaintext)] = self::URI . $hashtag->href;
// }
$item['content'] = '';
// avatar display
$item['content'] .= '<div style="float:left; margin: 0 0.5em 0.5em 0;"><a href="' . self::URI . urlencode($this->getInput('username'));
$item['content'] .= '"><img align="top" alt="avatar" src="' . $item['avatar'].'" />' . $item['username'] . '</a></div>';
$item['content'] .= '<div style="float:left; margin: 0 0.5em 0.5em 0;"><a href="'
. self::URI
. urlencode($this->getInput('username'));
$content = $post->find('div.Al', 0);
$item['content'] .= '"><img align="top" alt="'
. $item['author']
. '" src="'
. $item['avatar']
. '" /></a></div>';
// alter link
// $content = $content->innertext;
// $content = str_replace('href="./', 'href="' . self::URI, $content);
// $content = str_replace('href="photos', 'href="' . self::URI . 'photos', $content);
// XXX ugly but I don't have any idea how to do a better stuff, str_replace on link doesn't work as expected and ask too many checks
foreach($content->find('a') as $link)
{
$content = $post->find('div[jsname=EjRJtf]', 0);
// extract plaintext
$item['content_simple'] = $content->plaintext;
$item['title'] = substr($item['content_simple'], 0, 72) . '...';
// XXX ugly but I don't have any idea how to do a better stuff,
// str_replace on link doesn't work as expected and ask too many checks
foreach($content->find('a') as $link) {
$hasHttp = strpos($link->href, 'http');
$hasDoubleSlash = strpos($link->href, '//');
if ((!$hasHttp && !$hasDoubleSlash)
|| (false !== $hasHttp && strpos($link->href, 'http') != 0)
|| (false === $hasHttp && false !== $hasDoubleSlash && $hasDoubleSlash != 0))
{
if((!$hasHttp && !$hasDoubleSlash)
|| (false !== $hasHttp && strpos($link->href, 'http') != 0)
|| (false === $hasHttp && false !== $hasDoubleSlash && $hasDoubleSlash != 0)) {
// skipp bad link, for some hashtag or other stuff
if (strpos($link->href, '/') == 0)
{
if(strpos($link->href, '/') == 0) {
$link->href = substr($link->href, 1);
}
$link->href = self::URI . $link->href;
}
}
$content = $content->innertext;
$item['content'] .= '<div style="margin-top: -1.5em">' . $content . '</div>';
$item['content'] = trim(strip_tags($item['content'], '<a><p><div><img>'));
// extract plaintext
$item['content_simple'] = $post->find('div.Al', 0)->plaintext;
$this->items[] = $item;
}
// $html->save(__DIR__ . '/../posts2.html');
}
public function getName()
{
public function getName(){
return $this->_title ?: 'Google Plus Post Bridge';
}
public function getURI()
{
return $this->_url ?: self::URI;
public function getURI(){
return $this->_url ?: parent::getURI();
}
}

View File

@@ -7,48 +7,58 @@
* qdr:y : in past year
* sbd:1 : sort by date (will only work if qdr: is specified)
*/
class GoogleSearchBridge extends BridgeAbstract{
class GoogleSearchBridge extends BridgeAbstract {
const MAINTAINER = "sebsauvage";
const NAME = "Google search";
const URI = "https://www.google.com/";
const MAINTAINER = 'sebsauvage';
const NAME = 'Google search';
const URI = 'https://www.google.com/';
const CACHE_TIMEOUT = 1800; // 30min
const DESCRIPTION = "Returns most recent results from Google search.";
const DESCRIPTION = 'Returns most recent results from Google search.';
const PARAMETERS = array( array(
'q'=>array(
'name'=>"keyword",
'required'=>true
)
));
const PARAMETERS = array(array(
'q' => array(
'name' => 'keyword',
'required' => true
)
));
public function collectData(){
$html = '';
public function collectData(){
$html = '';
$html = getSimpleHTMLDOM(self::URI
. 'search?q='
. urlencode($this->getInput('q'))
.'&num=100&complete=0&tbs=qdr:y,sbd:1')
or returnServerError('No results for this query.');
$html = getSimpleHTMLDOM(self::URI
.'search?q=' . urlencode($this->getInput('q'))
.'&num=100&complete=0&tbs=qdr:y,sbd:1')
or returnServerError('No results for this query.');
$emIsRes = $html->find('div[id=ires]', 0);
$emIsRes = $html->find('div[id=ires]',0);
if( !is_null($emIsRes) ){
foreach($emIsRes->find('li[class=g]') as $element) {
$item = array();
if(!is_null($emIsRes)) {
foreach($emIsRes->find('div[class=g]') as $element) {
// Extract direct URL from google href (eg. /url?q=...)
$t = $element->find('a[href]',0)->href;
$item['uri'] = ''.$t;
parse_str(parse_url($t, PHP_URL_QUERY),$parameters);
if (isset($parameters['q'])) { $item['uri'] = $parameters['q']; }
$item['title'] = $element->find('h3',0)->plaintext;
$item['content'] = $element->find('span[class=st]',0)->plaintext;
$this->items[] = $item;
}
}
}
$item = array();
public function getName(){
return $this->getInput('q') .' - Google search';
}
// Extract direct URL from google href (eg. /url?q=...)
$t = $element->find('a[href]', 0)->href;
$item['uri'] = '' . $t;
parse_str(parse_url($t, PHP_URL_QUERY), $parameters);
if(isset($parameters['q'])) {
$item['uri'] = $parameters['q'];
}
$item['title'] = $element->find('h3', 0)->plaintext;
$item['content'] = $element->find('span[class=st]', 0)->plaintext;
$this->items[] = $item;
}
}
}
public function getName(){
if(!is_null($this->getInput('q'))) {
return $this->getInput('q') . ' - Google search';
}
return parent::getName();
}
}

View File

@@ -0,0 +1,61 @@
<?php
class GrandComicsDatabaseBridge extends BridgeAbstract {
const MAINTAINER = 'corenting';
const NAME = 'Grand Comics Database Bridge';
const URI = 'https://www.comics.org/';
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = 'Returns the latest comics added to a series timeline';
const PARAMETERS = array( array(
'series' => array(
'name' => 'Series id (from the timeline URL)',
'required' => true,
'exampleValue' => '63051',
),
));
public function collectData(){
$url = self::URI . 'series/' . $this->getInput('series') . '/details/timeline/';
$html = getSimpleHTMLDOM($url)
or returnServerError('Error while downloading the website content');
$table = $html->find('table', 0);
$list = array_reverse($table->find('[class^=row_even]'));
$seriesName = $html->find('span[id=series_name]', 0)->innertext;
// Get row headers
$rowHeaders = $table->find('th');
foreach($list as $article) {
// Skip empty rows
$emptyRow = $article->find('td.empty_month');
if (count($emptyRow) != 0) {
continue;
}
$rows = $article->find('td');
$key_date = $rows[0]->innertext;
// Get URL too
$uri = 'https://www.comics.org' . $article->find('a')[0]->href;
// Build content
$content = '';
for($i = 0; $i < count($rowHeaders); $i++) {
$headerItem = $rowHeaders[$i]->innertext;
$rowItem = $rows[$i]->innertext;
$content = $content . $headerItem . ': ' . $rowItem . '<br/>';
}
// Build final item
$item = array();
$item['title'] = $seriesName . ' - ' . $key_date;
$item['timestamp'] = strtotime($key_date);
$item['content'] = str_get_html($content);
$item['uri'] = $uri;
$this->items[] = $item;
}
}
}

View File

@@ -1,62 +1,83 @@
<?php
class HDWallpapersBridge extends BridgeAbstract {
const MAINTAINER = "nel50n";
const NAME = "HD Wallpapers Bridge";
const URI = "http://www.hdwallpapers.in/";
const MAINTAINER = 'nel50n';
const NAME = 'HD Wallpapers Bridge';
const URI = 'http://www.hdwallpapers.in/';
const CACHE_TIMEOUT = 43200; //12h
const DESCRIPTION = "Returns the latests wallpapers from HDWallpapers";
const DESCRIPTION = 'Returns the latests wallpapers from HDWallpapers';
const PARAMETERS = array( array(
'c'=>array(
'name'=>'category',
'defaultValue'=>'latest_wallpapers'
),
'm'=>array('name'=>'max number of wallpapers'),
'r'=>array(
'name'=>'resolution',
'defaultValue'=>'1920x1200',
'exampleValue'=>'1920x1200, 1680x1050,…'
)
));
const PARAMETERS = array( array(
'c' => array(
'name' => 'category',
'defaultValue' => 'latest_wallpapers'
),
'm' => array(
'name' => 'max number of wallpapers'
),
'r' => array(
'name' => 'resolution',
'defaultValue' => '1920x1200',
'exampleValue' => '1920x1200, 1680x1050,…'
)
));
public function collectData(){
$category = $this->category;
if (strrpos($category, 'wallpapers') !== strlen($category)-strlen('wallpapers')) {
$category .= '-desktop-wallpapers';
}
public function collectData(){
$category = $this->category;
if(strrpos($category, 'wallpapers') !== strlen($category) - strlen('wallpapers')) {
$category .= '-desktop-wallpapers';
}
$num = 0;
$max = $this->getInput('m') ?: 14;
$lastpage = 1;
$num = 0;
$max = $this->getInput('m') ?: 14;
$lastpage = 1;
for ($page = 1; $page <= $lastpage; $page++) {
$link = self::URI.'/'.$category.'/page/'.$page;
$html = getSimpleHTMLDOM($link) or returnServerError('No results for this query.');
for($page = 1; $page <= $lastpage; $page++) {
$link = self::URI . '/' . $category . '/page/' . $page;
$html = getSimpleHTMLDOM($link)
or returnServerError('No results for this query.');
if ($page === 1) {
preg_match('/page\/(\d+)$/', $html->find('.pagination a', -2)->href, $matches);
$lastpage = min($matches[1], ceil($max/14));
}
if($page === 1) {
preg_match('/page\/(\d+)$/', $html->find('.pagination a', -2)->href, $matches);
$lastpage = min($matches[1], ceil($max / 14));
}
foreach($html->find('.wallpapers .wall a') as $element) {
$thumbnail = $element->find('img', 0);
foreach($html->find('.wallpapers .wall a') as $element) {
$thumbnail = $element->find('img', 0);
$item = array();
// http://www.hdwallpapers.in/download/yosemite_reflections-1680x1050.jpg
$item['uri'] = self::URI.'/download'.str_replace('wallpapers.html', $this->getInput('r').'.jpg', $element->href);
$item['timestamp'] = time();
$item['title'] = $element->find('p', 0)->text();
$item['content'] = $item['title'].'<br><a href="'.$item['uri'].'"><img src="'.self::URI.$thumbnail->src.'" /></a>';
$this->items[] = $item;
$item = array();
// http://www.hdwallpapers.in/download/yosemite_reflections-1680x1050.jpg
$item['uri'] = self::URI
. '/download'
. str_replace('wallpapers.html', $this->getInput('r') . '.jpg', $element->href);
$num++;
if ($num >= $max)
break 2;
}
}
}
$item['timestamp'] = time();
$item['title'] = $element->find('p', 0)->text();
$item['content'] = $item['title']
. '<br><a href="'
. $item['uri']
. '"><img src="'
. self::URI
. $thumbnail->src
. '" /></a>';
public function getName(){
return 'HDWallpapers - '.str_replace(['__', '_'], [' & ', ' '], $this->getInput('c')).' ['.$this->getInput('r').']';
}
$this->items[] = $item;
$num++;
if ($num >= $max)
break 2;
}
}
}
public function getName(){
if(!is_null($this->getInput('c')) && !is_null($this->getInput('r'))) {
return 'HDWallpapers - '
. str_replace(['__', '_'], [' & ', ' '], $this->getInput('c'))
. ' ['
. $this->getInput('r')
. ']';
}
return parent::getName();
}
}

View File

@@ -1,23 +1,37 @@
<?php
class HentaiHavenBridge extends BridgeAbstract{
class HentaiHavenBridge extends BridgeAbstract {
const MAINTAINER = "albirew";
const NAME = "Hentai Haven";
const URI = "http://hentaihaven.org/";
const MAINTAINER = 'albirew';
const NAME = 'Hentai Haven';
const URI = 'http://hentaihaven.org/';
const CACHE_TIMEOUT = 21600; // 6h
const DESCRIPTION = "Returns releases from Hentai Haven";
const DESCRIPTION = 'Returns releases from Hentai Haven';
public function collectData(){
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not request Hentai Haven.');
foreach($html->find('div.zoe-grid') as $element) {
$item = array();
$item['uri'] = $element->find('div.brick-content h3 a', 0)->href;
$thumbnailUri = $element->find('a.thumbnail-image img', 0)->getAttribute('data-src');
$item['title'] = mb_convert_encoding(trim($element->find('div.brick-content h3 a', 0)->innertext), 'UTF-8', 'HTML-ENTITIES');
$item['tags'] = $element->find('div.oFlyout_bg div.oFlyout div.flyoutContent span.tags', 0)->plaintext;
$item['content'] = 'Tags: ' . $item['tags'].'<br><br><a href="' . $item['uri'] . '"><img width="300" height="169" src="' . $thumbnailUri . '" /></a><br>' . $element->find('div.oFlyout_bg div.oFlyout div.flyoutContent p.description', 0)->innertext;
$this->items[] = $item;
}
}
public function collectData(){
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not request Hentai Haven.');
foreach($html->find('div.zoe-grid') as $element) {
$item = array();
$item['uri'] = $element->find('div.brick-content h3 a', 0)->href;
$thumbnailUri = $element->find('a.thumbnail-image img', 0)->getAttribute('data-src');
$item['title'] = mb_convert_encoding(
trim($element->find('div.brick-content h3 a', 0)->innertext),
'UTF-8',
'HTML-ENTITIES'
);
$item['tags'] = $element->find('div.oFlyout_bg div.oFlyout div.flyoutContent span.tags', 0)->plaintext;
$item['content'] = 'Tags: '
. $item['tags']
. '<br><br><a href="'
. $item['uri']
. '"><img width="300" height="169" src="'
. $thumbnailUri
. '" /></a><br>'
. $element->find('div.oFlyout_bg div.oFlyout div.flyoutContent p.description', 0)->innertext;
$this->items[] = $item;
}
}
}

1397
bridges/HotUKDealsBridge.php Normal file

File diff suppressed because it is too large Load Diff

310
bridges/IPBBridge.php Normal file
View File

@@ -0,0 +1,310 @@
<?php
class IPBBridge extends FeedExpander {
const NAME = 'IPB Bridge';
const URI = 'https://www.invisionpower.com';
const DESCRIPTION = 'Returns feeds for forums powered by IPB';
const MAINTAINER = 'logmanoriginal';
const PARAMETERS = array(
array(
'uri' => array(
'name' => 'URI',
'type' => 'text',
'required' => true,
'title' => 'Insert forum, subforum or topic URI',
'exampleValue' => 'https://invisioncommunity.com/forums/forum/499-feedback-and-ideas/'
),
'limit' => array(
'name' => 'Limit',
'type' => 'number',
'required' => false,
'title' => 'Specifies the number of items to return on each request (-1: all)',
'defaultValue' => 10
)
)
);
const CACHE_TIMEOUT = 3600;
// Constants for internal use
const FORUM_TYPE_LIST_FILTER = '.cForumTopicTable';
const FORUM_TYPE_TABLE_FILTER = '#forum_table';
const TOPIC_TYPE_ARTICLE = 'article';
const TOPIC_TYPE_DIV = 'div.post_block';
public function getURI(){
return $this->getInput('uri') ?: parent::getURI();
}
public function collectData(){
// The URI cannot be the mainpage (or anything related)
switch(parse_url($this->getInput('uri'), PHP_URL_PATH)) {
case null:
case '/index.php':
returnClientError('Provided URI is invalid!');
break;
default:
break;
}
// Sanitize the URI (because else it won't work)
$uri = rtrim($this->getInput('uri'), '/'); // No trailing slashes!
// Forums might provide feeds, though that's optional *facepalm*
// Let's check if there is a valid feed available
$headers = get_headers($uri . '.xml');
if($headers[0] === 'HTTP/1.1 200 OK') { // Heureka! It's a valid feed!
return $this->collectExpandableDatas($uri);
}
// No valid feed, so do it the hard way
$html = getSimpleHTMLDOM($uri)
or returnServerError('Could not request ' . $this->getInput('uri') . '!');
$limit = $this->getInput('limit');
// Determine if this is a topic or a forum
switch(true) {
case $this->isTopic($html):
$this->collectTopic($html, $limit);
break;
case $this->isForum($html);
$this->collectForum($html);
break;
default:
returnClientError('Unknown type!');
break;
}
}
private function isForum($html){
return !is_null($html->find('div[data-controller*=forums.front.forum.forumPage]', 0))
|| !is_null($html->find(static::FORUM_TYPE_TABLE_FILTER, 0));
}
private function isTopic($html){
return !is_null($html->find('div[data-controller*=core.front.core.commentFeed]', 0))
|| !is_null($html->find(static::TOPIC_TYPE_DIV, 0));
}
private function collectForum($html){
// There are multiple forum designs in use (depends on version?)
// 1 - Uses an ordered list (based on https://invisioncommunity.com/forums)
// 2 - Uses a table (based on https://onehallyu.com)
switch(true) {
case !is_null($html->find(static::FORUM_TYPE_LIST_FILTER, 0)):
$this->collectForumList($html);
break;
case !is_null($html->find(static::FORUM_TYPE_TABLE_FILTER, 0)):
$this->collectForumTable($html);
break;
default:
returnClientError('Unknown forum format!');
break;
}
}
private function collectForumList($html){
foreach($html->find(static::FORUM_TYPE_LIST_FILTER, 0)->children() as $row) {
// Columns: Title, Statistics, Last modified
$item = array();
$item['uri'] = $row->find('a', 0)->href;
$item['title'] = $row->find('a', 0)->title;
$item['author'] = $row->find('a', 1)->innertext;
$item['timestamp'] = strtotime($row->find('time', 0)->getAttribute('datetime'));
$this->items[] = $item;
}
}
private function collectForumTable($html){
foreach($html->find(static::FORUM_TYPE_TABLE_FILTER, 0)->children() as $row) {
// Columns: Icon, Content, Preview, Statistics, Last modified
$item = array();
// Skip header row
if(!is_null($row->find('th', 0))) continue;
$item['uri'] = $row->find('a', 0)->href;
$item['title'] = $row->find('.title', 0)->plaintext;
$item['timestamp'] = strtotime($row->find('[itemprop=dateCreated]', 0)->plaintext);
$this->items[] = $item;
}
}
private function collectTopic($html, $limit){
// There are multiple topic designs in use (depends on version?)
// 1 - Uses articles (based on https://invisioncommunity.com/forums)
// 2 - Uses divs (based on https://onehallyu.com)
switch(true) {
case !is_null($html->find(static::TOPIC_TYPE_ARTICLE, 0)):
$this->collectTopicHistory($html, $limit, 'collectTopicArticle');
break;
case !is_null($html->find(static::TOPIC_TYPE_DIV, 0)):
$this->collectTopicHistory($html, $limit, 'collectTopicDiv');
break;
default:
returnClientError('Unknown topic format!');
break;
}
}
private function collectTopicHistory($html, $limit, $callback){
// Make sure the callback is valid!
if(!method_exists($this, $callback))
returnServerError('Unknown function (\'' . $callback . '\')!');
$next = null; // Holds the URI of the next page
while(true) {
$next = $this->$callback($html, is_null($next));
if(is_null($next) || ($limit > 0 && count($this->items) >= $limit)) {
break;
}
$html = getSimpleHTMLDOMCached($next);
}
// We might have more items than specified, remove excess
$this->items = array_slice($this->items, 0, $limit);
}
private function collectTopicArticle($html, $firstrun = true){
$title = $html->find('h1.ipsType_pageTitle', 0)->plaintext;
// Are we on last page?
if($firstrun && !is_null($html->find('.ipsPagination', 0))) {
$last = $html->find('.ipsPagination_last a', 0)->{'data-page'};
$active = $html->find('.ipsPagination_active a', 0)->{'data-page'};
if($active !== $last) {
// Load last page into memory (cached)
$html = getSimpleHTMLDOMCached($html->find('.ipsPagination_last a', 0)->href);
}
}
foreach(array_reverse($html->find(static::TOPIC_TYPE_ARTICLE)) as $article) {
$item = array();
$item['uri'] = $article->find('time', 0)->parent()->href;
$item['author'] = $article->find('aside a', 0)->plaintext;
$item['title'] = $item['author'] . ' - ' . $title;
$item['timestamp'] = strtotime($article->find('time', 0)->getAttribute('datetime'));
$content = $article->find('[data-role=commentContent]', 0);
$content = $this->scaleImages($content);
$item['content'] = $this->fixContent($content);
$item['enclosures'] = $this->findImages($article->find('[data-role=commentContent]', 0)) ?: null;
$this->items[] = $item;
}
// Return whatever page comes next (previous, as we add in inverse order)
// Do we have a previous page? (inactive means no)
if(!is_null($html->find('li[class=ipsPagination_prev ipsPagination_inactive]', 0))) {
return null; // No, or no more
} elseif(!is_null($html->find('li[class=ipsPagination_prev]', 0))) {
return $html->find('.ipsPagination_prev a', 0)->href;
}
return null;
}
private function collectTopicDiv($html, $firstrun = true){
$title = $html->find('h1.ipsType_pagetitle', 0)->plaintext;
// Are we on last page?
if($firstrun && !is_null($html->find('.pagination', 0))) {
$active = $html->find('li[class=page active]', 0)->plaintext;
// There are two ways the 'last' page is displayed:
// - With a distict 'last' button (only if there are enough pages)
// - With a button for each page (use last button)
if(!is_null($html->find('li.last', 0))) {
$last = $html->find('li.last a', 0);
} else {
$last = $html->find('li[class=page] a', -1);
}
if($active !== $last->plaintext) {
// Load last page into memory (cached)
$html = getSimpleHTMLDOMCached($last->href);
}
}
foreach(array_reverse($html->find(static::TOPIC_TYPE_DIV)) as $article) {
$item = array();
$item['uri'] = $article->find('a[rel=bookmark]', 0)->href;
$item['author'] = $article->find('.author', 0)->plaintext;
$item['title'] = $item['author'] . ' - ' . $title;
$item['timestamp'] = strtotime($article->find('.published', 0)->getAttribute('title'));
$content = $article->find('[itemprop=commentText]', 0);
$content = $this->scaleImages($content);
$item['content'] = $this->fixContent($content);
$item['enclosures'] = $this->findImages($article->find('.post_body', 0)) ?: null;
$this->items[] = $item;
}
// Return whatever page comes next (previous, as we add in inverse order)
// Do we have a previous page?
if(!is_null($html->find('li.prev', 0))) {
return $html->find('li.prev a', 0)->href;
}
return null;
}
/** Returns all images from the provide HTML DOM */
private function findImages($html){
$images = array();
foreach($html->find('img') as $img) {
$images[] = $img->src;
}
return $images;
}
/** Sets the maximum width and height for all images */
private function scaleImages($html, $width = 400, $height = 400){
foreach($html->find('img') as $img) {
$img->style = "max-width: {$width}px; max-height: {$height}px;";
}
return $html;
}
/** Removes all unnecessary tags and adds formatting */
private function fixContent($html){
// Restore quote highlighting
foreach($html->find('blockquote') as $quote) {
$quote->style = <<<EOD
padding: 0px 15px;
border-width: 1px 1px 1px 2px;
border-style: solid;
border-color: #ededed #e8e8e8 #dbdbdb #666666;
background: #fbfbfb;
EOD;
}
// Remove unnecessary tags
$content = strip_tags(
$html->innertext,
'<p><a><img><ol><ul><li><table><tr><th><td><strong><blockquote><br><hr><h>'
);
return $content;
}
}

View File

@@ -1,38 +1,52 @@
<?php
class IdenticaBridge extends BridgeAbstract{
class IdenticaBridge extends BridgeAbstract {
const MAINTAINER = "mitsukarenai";
const NAME = "Identica Bridge";
const URI = "https://identi.ca/";
const MAINTAINER = 'mitsukarenai';
const NAME = 'Identica Bridge';
const URI = 'https://identi.ca/';
const CACHE_TIMEOUT = 300; // 5min
const DESCRIPTION = "Returns user timelines";
const DESCRIPTION = 'Returns user timelines';
const PARAMETERS = array( array(
'u'=>array(
'name'=>'username',
'required'=>true
)
));
const PARAMETERS = array( array(
'u' => array(
'name' => 'username',
'required' => true
)
));
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Requested username can\'t be found.');
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Requested username can\'t be found.');
foreach($html->find('li.major') as $dent) {
$item = array();
$item['uri'] = html_entity_decode($dent->find('a', 0)->href); // get dent link
$item['timestamp'] = strtotime($dent->find('abbr.easydate', 0)->plaintext); // extract dent timestamp
$item['content'] = trim($dent->find('div.activity-content', 0)->innertext); // extract dent text
$item['title'] = $this->getInput('u') . ' | ' . $item['content'];
$this->items[] = $item;
}
}
foreach($html->find('li.major') as $dent) {
$item = array();
public function getName(){
return $this->getInput('u') .' - Identica Bridge';
}
// get dent link
$item['uri'] = html_entity_decode($dent->find('a', 0)->href);
public function getURI(){
return self::URI.urlencode($this->getInput('u'));
}
// extract dent timestamp
$item['timestamp'] = strtotime($dent->find('abbr.easydate', 0)->plaintext);
// extract dent text
$item['content'] = trim($dent->find('div.activity-content', 0)->innertext);
$item['title'] = $this->getInput('u') . ' | ' . $item['content'];
$this->items[] = $item;
}
}
public function getName(){
if(!is_null($this->getInput('u'))) {
return $this->getInput('u') . ' - Identica Bridge';
}
return parent::getName();
}
public function getURI(){
if(!is_null($this->getInput('u'))) {
return self::URI . urlencode($this->getInput('u'));
}
return parent::getURI();
}
}

View File

@@ -1,71 +1,149 @@
<?php
class InstagramBridge extends BridgeAbstract{
class InstagramBridge extends BridgeAbstract {
const MAINTAINER = "pauder";
const NAME = "Instagram Bridge";
const URI = "http://instagram.com/";
const DESCRIPTION = "Returns the newest images";
const MAINTAINER = 'pauder';
const NAME = 'Instagram Bridge';
const URI = 'https://instagram.com/';
const DESCRIPTION = 'Returns the newest images';
const PARAMETERS = array( array(
'u'=>array(
'name'=>'username',
'required'=>true
)
));
const PARAMETERS = array(
array(
'u' => array(
'name' => 'username',
'required' => true
)
),
array(
'h' => array(
'name' => 'hashtag',
'required' => true
)
),
'global' => array(
'media_type' => array(
'name' => 'Media type',
'type' => 'list',
'required' => false,
'values' => array(
'All' => 'all',
'Story' => 'story',
'Video' => 'video',
'Picture' => 'picture',
),
'defaultValue' => 'all'
)
)
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request Instagram.');
);
$innertext = null;
public function collectData(){
foreach($html->find('script') as $script)
{
if ('' === $script->innertext) {
continue;
}
if(!is_null($this->getInput('h')) && $this->getInput('media_type') == 'story') {
returnClientError('Stories are not supported for hashtags!');
}
$pos = strpos(trim($script->innertext), 'window._sharedData');
if (0 !== $pos)
{
continue;
}
$data = $this->getInstagramJSON($this->getURI());
$innertext = $script->innertext;
break;
}
if(!is_null($this->getInput('u'))) {
$userMedia = $data->entry_data->ProfilePage[0]->graphql->user->edge_owner_to_timeline_media->edges;
} else {
$userMedia = $data->entry_data->TagPage[0]->graphql->hashtag->edge_hashtag_to_media->edges;
}
$json = trim(substr($innertext, $pos+18), ' =;');
$data = json_decode($json);
foreach($userMedia as $media) {
$media = $media->node;
if(!is_null($this->getInput('u'))) {
switch($this->getInput('media_type')) {
case 'all': break;
case 'video':
if($media->__typename != 'GraphVideo') continue 2;
break;
case 'picture':
if($media->__typename != 'GraphImage') continue 2;
break;
case 'story':
if($media->__typename != 'GraphSidecar') continue 2;
break;
default: break;
}
} else {
if($this->getInput('media_type') == 'video' && !$media->is_video) continue;
}
$item = array();
$item['uri'] = self::URI . 'p/' . $media->shortcode . '/';
$userMedia = $data->entry_data->ProfilePage[0]->user->media->nodes;
if (isset($media->edge_media_to_caption->edges[0]->node->text)) {
$item['title'] = $media->edge_media_to_caption->edges[0]->node->text;
} else {
$item['title'] = basename($media->display_url);
}
foreach($userMedia as $media)
{
if(!is_null($this->getInput('u')) && $media->__typename == 'GraphSidecar') {
$data = $this->getInstagramStory($item['uri']);
$item['content'] = $data[0];
$item['enclosures'] = $data[1];
} else {
$item['content'] = '<img src="' . htmlentities($media->display_url) . '" alt="'. $item['title'] . '" />';
$item['enclosures'] = array($media->display_url);
}
$item = array();
$item['uri'] = self::URI.'p/'.$media->code.'/';
$item['content'] = '<img src="' . htmlentities($media->display_src) . '" />';
if (isset($media->caption))
{
$item['title'] = $media->caption;
} else {
$item['title'] = basename($media->display_src);
}
$item['timestamp'] = $media->date;
$this->items[] = $item;
$item['timestamp'] = $media->taken_at_timestamp;
}
}
$this->items[] = $item;
}
}
public function getName(){
return $this->getInput('u') .' - Instagram Bridge';
}
protected function getInstagramStory($uri) {
public function getURI(){
return self::URI.urlencode($this->getInput('u'));
}
$data = $this->getInstagramJSON($uri);
$mediaInfo = $data->entry_data->PostPage[0]->graphql->shortcode_media;
//Process the first element, that isn't in the node graph
$caption = $mediaInfo->edge_media_to_caption->edges[0]->node->text;
$enclosures = [$mediaInfo->display_url];
$content = '<img src="' . htmlentities($mediaInfo->display_url) . '" alt="'. $caption . '" />';
foreach($mediaInfo->edge_sidecar_to_children->edges as $media) {
$content .= '<img src="' . htmlentities($media->node->display_url) . '" alt="'. $caption . '" />';
$enclosures[] = $media->node->display_url;
}
return [$content, $enclosures];
}
protected function getInstagramJSON($uri) {
$html = getContents($uri)
or returnServerError('Could not request Instagram.');
$scriptRegex = '/window\._sharedData = (.*);<\/script>/';
preg_match($scriptRegex, $html, $matches, PREG_OFFSET_CAPTURE, 0);
return json_decode($matches[1][0]);
}
public function getName(){
if(!is_null($this->getInput('u'))) {
return $this->getInput('u') . ' - Instagram Bridge';
}
return parent::getName();
}
public function getURI(){
if(!is_null($this->getInput('u'))) {
return self::URI . urlencode($this->getInput('u'));
} elseif(!is_null($this->getInput('h'))) {
return self::URI . 'explore/tags/' . urlencode($this->getInput('h'));
}
return parent::getURI();
}
}

View File

@@ -1,468 +0,0 @@
<?php
class IsoHuntBridge extends BridgeAbstract{
const MAINTAINER = 'logmanoriginal';
const NAME = 'isoHunt Bridge';
const URI = 'https://isohunt.to/';
const CACHE_TIMEOUT = 300; //5min
const DESCRIPTION = 'Returns the latest results by category or search result';
const PARAMETERS = array(
/*
* Get feeds for one of the "latest" categories
* Notice: The categories "News" and "Top Searches" are received from the main page
* Elements are sorted by name ascending!
*/
'By "Latest" category' => array(
'latest_category'=>array(
'name'=>'Latest category',
'type'=>'list',
'required'=>true,
'title'=>'Select your category',
'defaultValue'=>'news',
'values'=>array(
'Hot Torrents'=>'hot_torrents',
'News'=>'news',
'Releases'=>'releases',
'Torrents'=>'torrents'
)
)
),
/*
* Get feeds for one of the "torrent" categories
* Make sure to add new categories also to get_torrent_category_index($)!
* Elements are sorted by name ascending!
*/
'By "Torrent" category' => array(
'torrent_category'=>array(
'name'=>'Torrent category',
'type'=>'list',
'required'=>true,
'title'=>'Select your category',
'defaultValue'=>'anime',
'values'=>array(
'Adult'=>'adult',
'Anime'=>'anime',
'Books'=>'books',
'Games'=>'games',
'Movies'=>'movies',
'Music'=>'music',
'Other'=>'other',
'Series & TV'=>'series_tv',
'Software'=>'software'
)
),
'torrent_popularity'=>array(
'name'=>'Sort by popularity',
'type'=>'checkbox',
'title'=>'Activate to receive results by popularity'
)
),
/*
* Get feeds for a specific search request
*/
'Search torrent by name' => array(
'search_name'=>array(
'name'=>'Name',
'required'=>true,
'title'=>'Insert your search query',
'exampleValue'=>'Bridge'
),
'search_category'=>array(
'name'=>'Category',
'type'=>'list',
'title'=>'Select your category',
'defaultValue'=>'all',
'values'=>array(
'Adult'=>'adult',
'All'=>'all',
'Anime'=>'anime',
'Books'=>'books',
'Games'=>'games',
'Movies'=>'movies',
'Music'=>'music',
'Other'=>'other',
'Series & TV'=>'series_tv',
'Software'=>'software'
)
)
)
);
public function getURI(){
$uri=self::URI;
switch($this->queriedContext){
case 'By "Latest" category':
switch($this->getInput('latest_category')){
case 'hot_torrents':
$uri .= 'statistic/hot/torrents';
break;
case 'news':
break;
case 'releases':
$uri .= 'releases.php';
break;
case 'torrents':
$uri .= 'latest.php';
break;
}
break;
case 'By "Torrent" category':
$uri .= $this->build_category_uri(
$this->getInput('torrent_category'),
$this->getInput('torrent_popularity')
);
break;
case 'Search torrent by name':
$category=$this->getInput('search_category');
$uri .= $this->build_category_uri($category);
if($category!=='movies')
$uri .= '&ihq=' . urlencode($this->getInput('search_name'));
break;
}
return $uri;
}
public function getName(){
switch($this->queriedContext){
case 'By "Latest" category':
$categoryName =
array_search(
$this->getInput('latest_category'),
self::PARAMETERS['By "Latest" category']['latest_category']['values']
);
$name = 'Latest '.$categoryName.' - ' . self::NAME;
break;
case 'By "Torrent" category':
$categoryName =
array_search(
$this->getInput('torrent_category'),
self::PARAMETERS['By "Torrent" category']['torrent_category']['values']
);
$name = 'Category: ' . $categoryName . ' - ' . self::NAME;
break;
case 'Search torrent by name':
$categoryName =
array_search(
$this->getInput('search_category'),
self::PARAMETERS['Search torrent by name']['search_category']['values']
);
$name = 'Search: "' . $this->getInput('search_name') . '" in category: ' . $categoryName . ' - ' . self::NAME;
break;
}
return $name;
}
public function collectData(){
$html = $this->load_html($this->getURI());
switch($this->queriedContext){
case 'By "Latest" category':
switch($this->getInput('latest_category')){
case 'hot_torrents':
$this->get_latest_hot_torrents($html);
break;
case 'news':
$this->get_latest_news($html);
break;
case 'releases':
case 'torrents':
$this->get_latest_torrents($html);
break;
}
break;
case 'By "Torrent" category':
if($this->getInput('torrent_category') === 'movies'){
// This one is special (content wise)
$this->get_movie_torrents($html);
}else{
$this->get_latest_torrents($html);
}
break;
case 'Search torrent by name':
if( $this->getInput('search_category') === 'movies'){
// This one is special (content wise)
$this->get_movie_torrents($html);
} else {
$this->get_latest_torrents($html);
}
break;
}
}
#region Helper functions for "Movie Torrents"
private function get_movie_torrents($html){
$container = $html->find('div#w0', 0);
if(!$container)
returnServerError('Unable to find torrent container!');
$torrents = $container->find('article');
if(!$torrents)
returnServerError('Unable to find torrents!');
foreach($torrents as $torrent){
$anchor = $torrent->find('a', 0);
if(!$anchor)
returnServerError('Unable to find anchor!');
$date = $torrent->find('small', 0);
if(!$date)
returnServerError('Unable to find date!');
$item = array();
$item['uri'] = $this->fix_relative_uri($anchor->href);
$item['title'] = $anchor->title;
// $item['author'] =
$item['timestamp'] = strtotime($date->plaintext);
$item['content'] = $this->fix_relative_uri($torrent->innertext);
$this->items[] = $item;
}
}
#endregion
#region Helper functions for "Latest Hot Torrents"
private function get_latest_hot_torrents($html){
$container = $html->find('div#serps', 0);
if(!$container)
returnServerError('Unable to find torrent container!');
$torrents = $container->find('tr');
if(!$torrents)
returnServerError('Unable to find torrents!');
// Remove first element (header row)
$torrents = array_slice($torrents, 1);
foreach($torrents as $torrent){
$cell = $torrent->find('td', 0);
if(!$cell)
returnServerError('Unable to find cell!');
$element = $cell->find('a', 0);
if(!$element)
returnServerError('Unable to find element!');
$item = array();
$item['uri'] = $element->href;
$item['title'] = $element->plaintext;
// $item['author'] =
// $item['timestamp'] =
// $item['content'] =
$this->items[] = $item;
}
}
#endregion
#region Helper functions for "Latest News"
private function get_latest_news($html){
$container = $html->find('div#postcontainer', 0);
if(!$container)
returnServerError('Unable to find post container!');
$posts = $container->find('div.index-post');
if(!$posts)
returnServerError('Unable to find posts!');
foreach($posts as $post){
$item = array();
$item['uri'] = $this->latest_news_extract_uri($post);
$item['title'] = $this->latest_news_extract_title($post);
$item['author'] = $this->latest_news_extract_author($post);
$item['timestamp'] = $this->latest_news_extract_timestamp($post);
$item['content'] = $this->latest_news_extract_content($post);
$this->items[] = $item;
}
}
private function latest_news_extract_author($post){
$author = $post->find('small', 0);
if(!$author)
returnServerError('Unable to find author!');
// The author is hidden within a string like: 'Posted by {author} on {date}'
preg_match('/Posted\sby\s(.*)\son/i', $author->innertext, $matches);
return $matches[1];
}
private function latest_news_extract_timestamp($post){
$date = $post->find('small', 0);
if(!$date)
returnServerError('Unable to find date!');
// The date is hidden within a string like: 'Posted by {author} on {date}'
preg_match('/Posted\sby\s.*\son\s(.*)/i', $date->innertext, $matches);
$timestamp = strtotime($matches[1]);
// Make sure date is not in the future (dates are given like 'Nov. 20' without year)
if($timestamp > time()){
$timestamp = strtotime('-1 year', $timestamp);
}
return $timestamp;
}
private function latest_news_extract_title($post){
$title = $post->find('a', 0);
if(!$title)
returnServerError('Unable to find title!');
return $title->plaintext;
}
private function latest_news_extract_uri($post){
$uri = $post->find('a', 0);
if(!$uri)
returnServerError('Unable to find uri!');
return $uri->href;
}
private function latest_news_extract_content($post){
$content = $post->find('div', 0);
if(!$content)
returnServerError('Unable to find content!');
// Remove <h2>...</h2> (title)
foreach($content->find('h2') as $element){
$element->outertext = '';
}
// Remove <small>...</small> (author)
foreach($content->find('small') as $element){
$element->outertext = '';
}
return $content->innertext;
}
#endregion
#region Helper functions for "Latest Torrents", "Latest Releases" and "Torrent Category"
private function get_latest_torrents($html){
$container = $html->find('div#serps', 0);
if(!$container)
returnServerError('Unable to find torrent container!');
$torrents = $container->find('tr[data-key]');
if(!$torrents)
returnServerError('Unable to find torrents!');
foreach($torrents as $torrent){
$item = array();
$item['uri'] = $this->latest_torrents_extract_uri($torrent);
$item['title'] = $this->latest_torrents_extract_title($torrent);
$item['author'] = $this->latest_torrents_extract_author($torrent);
$item['timestamp'] = $this->latest_torrents_extract_timestamp($torrent);
$item['content'] = ''; // There is no valuable content
$this->items[] = $item;
}
}
private function latest_torrents_extract_title($torrent){
$cell = $torrent->find('td.title-row', 0);
if(!$cell)
returnServerError('Unable to find title cell!');
$title = $cell->find('span', 0);
if(!$title)
returnServerError('Unable to find title!');
return $title->plaintext;
}
private function latest_torrents_extract_uri($torrent){
$cell = $torrent->find('td.title-row', 0);
if(!$cell)
returnServerError('Unable to find title cell!');
$uri = $cell->find('a', 0);
if(!$uri)
returnServerError('Unable to find uri!');
return $this->fix_relative_uri($uri->href);
}
private function latest_torrents_extract_author($torrent){
$cell = $torrent->find('td.user-row', 0);
if(!$cell)
return; // No author
$user = $cell->find('a', 0);
if(!$user)
returnServerError('Unable to find user!');
return $user->plaintext;
}
private function latest_torrents_extract_timestamp($torrent){
$cell = $torrent->find('td.date-row', 0);
if(!$cell)
returnServerError('Unable to find date cell!');
return strtotime('-' . $cell->plaintext, time());
}
#endregion
#region Generic helper functions
private function load_html($uri){
$html = getSimpleHTMLDOM($uri);
if(!$html)
returnServerError('Unable to load ' . $uri . '!');
return $html;
}
private function fix_relative_uri($uri){
return preg_replace('/\//i', self::URI, $uri, 1);
}
private function build_category_uri($category, $order_popularity = false){
switch($category){
case 'anime': $index = 1; break;
case 'software' : $index = 2; break;
case 'games' : $index = 3; break;
case 'adult' : $index = 4; break;
case 'movies' : $index = 5; break;
case 'music' : $index = 6; break;
case 'other' : $index = 7; break;
case 'series_tv' : $index = 8; break;
case 'books': $index = 9; break;
case 'all':
default: $index = 0; break;
}
return 'torrents/?iht=' . $index . '&ihs=' . ($order_popularity ? 1 : 0) . '&age=0';
}
#endregion
}

View File

@@ -1,89 +1,100 @@
<?php
class JapanExpoBridge extends BridgeAbstract {
const MAINTAINER = 'Ginko';
const NAME = 'Japan Expo Actualités';
const URI = 'http://www.japan-expo-paris.com/fr/actualites';
const CACHE_TIMEOUT = 14400; // 4h
const DESCRIPTION = 'Returns most recent entries from Japan Expo actualités.';
const PARAMETERS = array( array(
'mode'=>array(
'name'=>'Show full contents',
'type'=>'checkbox',
)
));
const MAINTAINER = 'Ginko';
const NAME = 'Japan Expo Actualités';
const URI = 'http://www.japan-expo-paris.com/fr/actualites';
const CACHE_TIMEOUT = 14400; // 4h
const DESCRIPTION = 'Returns most recent entries from Japan Expo actualités.';
const PARAMETERS = array( array(
'mode' => array(
'name' => 'Show full contents',
'type' => 'checkbox',
)
));
public function collectData(){
public function collectData(){
function french_pubdate_to_timestamp($date_to_parse) {
return strtotime(
strtr(
strtolower(str_replace('Publié le ', '', $date_to_parse)),
array(
'janvier' => 'jan',
'février' => 'feb',
'mars' => 'march',
'avril' => 'apr',
'mai' => 'may',
'juin' => 'jun',
'juillet' => 'jul',
'août' => 'aug',
'septembre' => 'sep',
'octobre' => 'oct',
'novembre' => 'nov',
'décembre' => 'dec'
)
)
);
}
function frenchPubDateToTimestamp($date_to_parse) {
return strtotime(
strtr(
strtolower(str_replace('Publié le ', '', $date_to_parse)),
array(
'janvier' => 'jan',
'février' => 'feb',
'mars' => 'march',
'avril' => 'apr',
'mai' => 'may',
'juin' => 'jun',
'juillet' => 'jul',
'août' => 'aug',
'septembre' => 'sep',
'octobre' => 'oct',
'novembre' => 'nov',
'décembre' => 'dec'
)
)
);
}
$convert_article_images = function ($matches) {
if (is_array($matches) && count($matches) > 1) {
return '<img src="'.$matches[1].'" />';
}
};
$convert_article_images = function($matches){
if(is_array($matches) && count($matches) > 1) {
return '<img src="' . $matches[1] . '" />';
}
};
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not request JapanExpo: '.self::URI);
$fullcontent = $this->getInput('mode');
$count = 0;
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not request JapanExpo: ' . self::URI);
$fullcontent = $this->getInput('mode');
$count = 0;
foreach ($html->find('a._tile2') as $element) {
foreach($html->find('a._tile2') as $element) {
$url = $element->href;
$thumbnail = 'http://s.japan-expo.com/katana/images/JES049/paris.png';
preg_match('/url\(([^)]+)\)/', $element->find('img.rspvimgset', 0)->style, $img_search_result);
if (count($img_search_result) >= 2)
$thumbnail = trim($img_search_result[1], "'");
$url = $element->href;
$thumbnail = 'http://s.japan-expo.com/katana/images/JES049/paris.png';
preg_match('/url\(([^)]+)\)/', $element->find('img.rspvimgset', 0)->style, $img_search_result);
if ($fullcontent) {
if ($count >= 5) {
break;
}
if(count($img_search_result) >= 2)
$thumbnail = trim($img_search_result[1], "'");
$article_html = getSimpleHTMLDOMCached('Could not request JapanExpo: '.$url);
$header = $article_html->find('header.pageHeadBox', 0);
$timestamp = strtotime($header->find('time', 0)->datetime);
$title_html = $header->find('div.section', 0)->next_sibling();
$title = $title_html->plaintext;
$headings = $title_html->next_sibling()->outertext;
$article = $article_html->find('div.content', 0)->innertext;
$article = preg_replace_callback('/<img [^>]+ style="[^\(]+\(\'([^\']+)\'[^>]+>/i', $convert_article_images, $article);
$content = $headings.$article;
} else {
$date_text = $element->find('span.date', 0)->plaintext;
$timestamp = french_pubdate_to_timestamp($date_text);
$title = trim($element->find('span._title', 0)->plaintext);
$content = '<img src="'.$thumbnail.'"></img><br />'.$date_text.'<br /><a href="'.$url.'">Lire l\'article</a>';
}
if($fullcontent) {
if($count >= 5) {
break;
}
$item = array();
$item['uri'] = $url;
$item['title'] = $title;
$item['timestamp'] = $timestamp;
$item['content'] = $content;
$this->items[] = $item;
$count++;
}
}
$article_html = getSimpleHTMLDOMCached('Could not request JapanExpo: ' . $url);
$header = $article_html->find('header.pageHeadBox', 0);
$timestamp = strtotime($header->find('time', 0)->datetime);
$title_html = $header->find('div.section', 0)->next_sibling();
$title = $title_html->plaintext;
$headings = $title_html->next_sibling()->outertext;
$article = $article_html->find('div.content', 0)->innertext;
$article = preg_replace_callback(
'/<img [^>]+ style="[^\(]+\(\'([^\']+)\'[^>]+>/i',
$convert_article_images,
$article);
$content = $headings . $article;
} else {
$date_text = $element->find('span.date', 0)->plaintext;
$timestamp = frenchPubDateToTimestamp($date_text);
$title = trim($element->find('span._title', 0)->plaintext);
$content = '<img src="'
. $thumbnail
. '"></img><br />'
. $date_text
. '<br /><a href="'
. $url
. '">Lire l\'article</a>';
}
$item = array();
$item['uri'] = $url;
$item['title'] = $title;
$item['timestamp'] = $timestamp;
$item['content'] = $content;
$this->items[] = $item;
$count++;
}
}
}

353
bridges/JustETFBridge.php Normal file
View File

@@ -0,0 +1,353 @@
<?php
class JustETFBridge extends BridgeAbstract {
const NAME = 'justETF Bridge';
const URI = 'https://www.justetf.com';
const DESCRIPTION = 'Currently only supports the news feed';
const MAINTAINER = 'logmanoriginal';
const PARAMETERS = array(
'News' => array(
'full' => array(
'name' => 'Full Article',
'type' => 'checkbox',
'title' => 'Enable to load full articles'
)
),
'Profile' => array(
'isin' => array(
'name' => 'ISIN',
'type' => 'text',
'required' => true,
'pattern' => '[a-zA-Z]{2}[a-zA-Z0-9]{10}',
'title' => 'ISIN, consisting of 2-letter country code, 9-character identifier, check character'
),
'strategy' => array(
'name' => 'Include Strategy',
'type' => 'checkbox',
'defaultValue' => 'checked'
),
'description' => array(
'name' => 'Include Description',
'type' => 'checkbox',
'defaultValue' => 'checked'
)
),
'global' => array(
'lang' => array(
'name' => 'Language',
'required' => true,
'type' => 'list',
'values' => array(
'Englisch' => 'en',
'Deutsch' => 'de',
'Italiano' => 'it'
),
'defaultValue' => 'Englisch'
)
)
);
public function collectData() {
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Failed loading contents from ' . $this->getURI());
defaultLinkTo($html, static::URI);
switch($this->queriedContext) {
case 'News':
$this->collectNews($html);
break;
case 'Profile':
$this->collectProfile($html);
break;
}
}
public function getURI() {
$uri = static::URI;
if($this->getInput('lang')) {
$uri .= '/' . $this->getInput('lang');
}
switch($this->queriedContext) {
case 'News':
$uri .= '/news';
break;
case 'Profile':
$uri .= '/etf-profile.html?' . http_build_query(array(
'isin' => strtoupper($this->getInput('isin'))
));
break;
}
return $uri;
}
public function getName() {
$name = static::NAME;
$name .= ($this->queriedContext) ? ' - ' . $this->queriedContext : '';
switch($this->queriedContext) {
case 'News': break;
case 'Profile':
if($this->getInput('isin')) {
$name .= ' ISIN ' . strtoupper($this->getInput('isin'));
}
}
if($this->getInput('lang')) {
$name .= ' (' . strtoupper($this->getInput('lang')) . ')';
}
return $name;
}
#region Common
/**
* Fixes dates depending on the choosen language:
*
* de : dd.mm.yy
* en : dd.mm.yy
* it : dd/mm/yy
*
* Basically strtotime doesn't convert dates correctly due to formats
* being hard to interpret. So we use the DateTime object, manually
* fixing dates and times (set to 00:00:00.000).
*
* We don't know the timezone, so just assume +00:00 (or whatever
* DateTime chooses)
*/
private function fixDate($date) {
switch($this->getInput('lang')) {
case 'en':
case 'de':
$df = date_create_from_format('d.m.y', $date);
break;
case 'it':
$df = date_create_from_format('d/m/y', $date);
break;
}
date_time_set($df, 0, 0);
// debugMessage(date_format($df, 'U'));
return date_format($df, 'U');
}
private function extractImages($article) {
// Notice: We can have zero or more images (though it should mostly be 1)
$elements = $article->find('img');
$images = array();
foreach($elements as $img) {
// Skip the logo (mostly provided part of a hidden div)
if(substr($img->src, strrpos($img->src, '/') + 1) === 'logo.png')
continue;
$images[] = $img->src;
}
return $images;
}
#endregion
#region News
private function collectNews($html) {
$articles = $html->find('div.newsTopArticle')
or returnServerError('No articles found! Layout might have changed!');
foreach($articles as $article) {
$item = array();
// Common data
$item['uri'] = $this->extractNewsUri($article);
$item['timestamp'] = $this->extractNewsDate($article);
$item['title'] = $this->extractNewsTitle($article);
if($this->getInput('full')) {
$uri = $this->extractNewsUri($article);
$html = getSimpleHTMLDOMCached($uri)
or returnServerError('Failed loading full article from ' . $uri);
$fullArticle = $html->find('div.article', 0)
or returnServerError('No content found! Layout might have changed!');
defaultLinkTo($fullArticle, static::URI);
$item['author'] = $this->extractFullArticleAuthor($fullArticle);
$item['content'] = $this->extractFullArticleContent($fullArticle);
$item['enclosures'] = $this->extractImages($fullArticle);
} else {
$item['content'] = $this->extractNewsDescription($article);
$item['enclosures'] = $this->extractImages($article);
}
$this->items[] = $item;
}
}
private function extractNewsUri($article) {
$element = $article->find('a', 0)
or returnServerError('Anchor not found!');
return $element->href;
}
private function extractNewsDate($article) {
$element = $article->find('div.subheadline', 0)
or returnServerError('Date not found!');
// debugMessage($element->plaintext);
$date = trim(explode('|', $element->plaintext)[0]);
return $this->fixDate($date);
}
private function extractNewsDescription($article) {
$element = $article->find('span.newsText', 0)
or returnServerError('Description not found!');
$element->find('a', 0)->onclick = '';
// debugMessage($element->innertext);
return $element->innertext;
}
private function extractNewsTitle($article) {
$element = $article->find('h3', 0)
or returnServerError('Title not found!');
return $element->plaintext;
}
private function extractFullArticleContent($article) {
$element = $article->find('div.article_body', 0)
or returnServerError('Article body not found!');
// Remove teaser image
$element->find('img.teaser-img', 0)->outertext = '';
// Remove self advertisements
foreach($element->find('.call-action') as $adv) {
$adv->outertext = '';
}
// Remove tips
foreach($element->find('.panel-edu') as $tip) {
$tip->outertext = '';
}
// Remove inline scripts (used for i.e. interactive graphs) as they are
// rendered as a long series of strings
foreach($element->find('script') as $script) {
$script->outertext = '[Content removed! Visit site to see full contents!]';
}
return $element->innertext;
}
private function extractFullArticleAuthor($article) {
$element = $article->find('span[itemprop=name]', 0)
or returnServerError('Author not found!');
return $element->plaintext;
}
#endregion
#region Profile
private function collectProfile($html) {
$item = array();
$item['uri'] = $this->getURI();
$item['timestamp'] = $this->extractProfileDate($html);
$item['title'] = $this->extractProfiletitle($html);
$item['author'] = $this->extractProfileAuthor($html);
$item['content'] = $this->extractProfileContent($html);
$this->items[] = $item;
}
private function extractProfileDate($html) {
$element = $html->find('div.infobox div.vallabel', 0)
or returnServerError('Date not found!');
// debugMessage($element->plaintext);
$date = trim(explode("\r\n", $element->plaintext)[1]);
return $this->fixDate($date);
}
private function extractProfileTitle($html) {
$element = $html->find('span.h1', 0)
or returnServerError('Title not found!');
return $element->plaintext;
}
private function extractProfileContent($html) {
// There are a few thins we are interested:
// - Investment Strategy
// - Description
// - Quote
$strategy = $html->find('div.tab-container div.col-sm-6 p', 0)
or returnServerError('Investment Strategy not found!');
// Description requires a bit of cleanup due to lack of propper identification
$description = $html->find('div.headline', 5)
or returnServerError('Description container not found!');
$description = $description->parent();
foreach($description->find('div') as $div) {
$div->outertext = '';
}
$quote = $html->find('div.infobox div.val', 0)
or returnServerError('Quote not found!');
$quote_html = '<strong>Quote</strong><br><p>' . $quote . '</p>';
$strategy_html = '';
$description_html = '';
if($this->getInput('strategy') === true) {
$strategy_html = '<strong>Strategy</strong><br><p>' . $strategy . '</p><br>';
}
if($this->getInput('description') === true) {
$description_html = '<strong>Description</strong><br><p>' . $description . '</p><br>';
}
return $strategy_html . $description_html . $quote_html;
}
private function extractProfileAuthor($html) {
// Use ISIN + WKN as author
// Notice: "identfier" is not a typo [sic]!
$element = $html->find('span.identfier', 0)
or returnServerError('Author not found!');
return $element->plaintext;
}
#endregion
}

123
bridges/KATBridge.php Normal file
View File

@@ -0,0 +1,123 @@
<?php
class KATBridge extends BridgeAbstract {
const MAINTAINER = 'niawag';
const NAME = 'KickassTorrents';
const URI = 'https://katcr.co/new/';
const DESCRIPTION = 'Returns results for the keywords. You can put several
list of keywords by separating them with a semicolon (e.g. "one show;another
show"). Category based search needs the category number as input. User based
search takes the Uploader ID: see KAT URL for user feed. Search can be done in a specified category';
const PARAMETERS = array( array(
'q' => array(
'name' => 'keywords, separated by semicolons',
'exampleValue' => 'first list;second list;…',
'required' => true
),
'crit' => array(
'type' => 'list',
'name' => 'Search type',
'values' => array(
'search' => 'search',
'category' => 'cat',
'user' => 'usr'
)
),
'cat_check' => array(
'type' => 'checkbox',
'name' => 'Specify category for normal search ?',
),
'cat' => array(
'name' => 'Category number',
'exampleValue' => '100, 200… See KAT for category number'
),
'trusted' => array(
'type' => 'checkbox',
'name' => 'Only get results from Elite or Verified uploaders ?',
),
));
public function collectData(){
function parseDateTimestamp($element){
$guessedDate = strptime($element, '%d-%m-%Y %H:%M:%S');
$timestamp = mktime(
$guessedDate['tm_hour'],
$guessedDate['tm_min'],
$guessedDate['tm_sec'],
$guessedDate['tm_mon'] + 1,
$guessedDate['tm_mday'],
$guessedDate['tm_year'] + 1900);
return $timestamp;
}
$catBool = $this->getInput('cat_check');
if($catBool) {
$catNum = $this->getInput('cat');
}
$critList = $this->getInput('crit');
$trustedBool = $this->getInput('trusted');
$keywordsList = explode(';', $this->getInput('q'));
foreach($keywordsList as $keywords) {
switch($critList) {
case 'search':
if($catBool == false) {
$html = getSimpleHTMLDOM(
self::URI .
'torrents-search.php?search=' .
rawurlencode($keywords)
) or returnServerError('Could not request KAT.');
} else {
$html = getSimpleHTMLDOM(
self::URI .
'torrents-search.php?search=' .
rawurlencode($keywords) .
'&cat=' .
rawurlencode($catNum)
) or returnServerError('Could not request KAT.');
}
break;
case 'cat':
$html = getSimpleHTMLDOM(
self::URI .
'torrents.php?cat=' .
rawurlencode($keywords)
) or returnServerError('Could not request KAT.');
break;
case 'usr':
$html = getSimpleHTMLDOM(
self::URI .
'account-details.php?id=' .
rawurlencode($keywords)
) or returnServerError('Could not request KAT.');
break;
}
if ($html->find('table.ttable_headinner', 0) == false)
returnServerError('No result for query ' . $keywords);
foreach($html->find('tr.t-row') as $element) {
if(!$trustedBool
|| !is_null($element->find('i[title="Elite Uploader"]', 0))
|| !is_null($element->find('i[title="Verified Uploader"]', 0))) {
$item = array();
$item['uri'] = self::URI . $element->find('a', 2)->href;
$item['id'] = self::URI . $element->find('a.cellMainLink', 0)->href;
$item['timestamp'] = parseDateTimestamp($element->find('td', 2)->plaintext);
$item['author'] = $element->find('a.plain', 0)->plaintext;
$item['title'] = $element->find('a.cellMainLink', 0)->plaintext;
$item['seeders'] = (int)$element->find('td', 3)->plaintext;
$item['leechers'] = (int)$element->find('td', 4)->plaintext;
$item['size'] = $element->find('td', 1)->plaintext;
$item['content'] = $item['title']
. '<br>size: '
. $item['size']
. '<br>seeders: '
. $item['seeders']
. ' | leechers: '
. $item['leechers']
. '<br><a href="'
. $item['id']
. '">info page</a>';
if(isset($item['title']))
$this->items[] = $item;
}
}
}
}
}

View File

@@ -0,0 +1,150 @@
<?php
class KernelBugTrackerBridge extends BridgeAbstract {
const NAME = 'Kernel Bug Tracker';
const URI = 'https://bugzilla.kernel.org';
const DESCRIPTION = 'Returns feeds for bug comments';
const MAINTAINER = 'logmanoriginal';
const PARAMETERS = array(
'Bug comments' => array(
'id' => array(
'name' => 'Bug tracking ID',
'type' => 'number',
'required' => true,
'title' => 'Insert bug tracking ID',
'exampleValue' => 121241
),
'limit' => array(
'name' => 'Number of comments to return',
'type' => 'number',
'required' => false,
'title' => 'Specify number of comments to return',
'defaultValue' => -1
),
'sorting' => array(
'name' => 'Sorting',
'type' => 'list',
'required' => false,
'title' => 'Defines the sorting order of the comments returned',
'defaultValue' => 'of',
'values' => array(
'Oldest first' => 'of',
'Latest first' => 'lf'
)
)
)
);
private $bugid = '';
private $bugdesc = '';
public function collectData(){
$limit = $this->getInput('limit');
$sorting = $this->getInput('sorting');
// We use the print preview page for simplicity
$html = getSimpleHTMLDOMCached($this->getURI() . '&format=multiple',
86400,
null,
null,
true,
true,
DEFAULT_TARGET_CHARSET,
false, // Do NOT remove line breaks
DEFAULT_BR_TEXT,
DEFAULT_SPAN_TEXT);
if($html === false)
returnServerError('Failed to load page!');
// Store header information into private members
$this->bugid = $html->find('#bugzilla-body', 0)->find('a', 0)->innertext;
$this->bugdesc = $html->find('table.bugfields', 0)->find('tr', 0)->find('td', 0)->innertext;
// Get and limit comments
$comments = $html->find('div.bz_comment');
if($limit > 0 && count($comments) > $limit) {
$comments = array_slice($comments, count($comments) - $limit, $limit);
}
// Order comments
switch($sorting) {
case 'lf': $comments = array_reverse($comments, true);
case 'of':
default: // Nothing to do, keep original order
}
foreach($comments as $comment) {
$comment = $this->inlineStyles($comment);
$item = array();
$item['uri'] = $this->getURI() . '#' . $comment->id;
$item['author'] = $comment->find('span.bz_comment_user', 0)->innertext;
$item['title'] = $comment->find('span.bz_comment_number', 0)->find('a', 0)->innertext;
$item['timestamp'] = strtotime($comment->find('span.bz_comment_time', 0)->innertext);
$item['content'] = $comment->find('pre.bz_comment_text', 0)->innertext;
// Fix line breaks (they use LF)
$item['content'] = str_replace("\n", '<br>', $item['content']);
// Fix relative URIs
$item['content'] = $this->replaceRelativeURI($item['content']);
$this->items[] = $item;
}
}
public function getURI(){
switch($this->queriedContext) {
case 'Bug comments':
return parent::getURI()
. '/show_bug.cgi?id='
. $this->getInput('id');
break;
default: return parent::getURI();
}
}
public function getName(){
switch($this->queriedContext) {
case 'Bug comments':
return 'Bug '
. $this->bugid
. ' tracker for '
. $this->bugdesc
. ' - '
. parent::getName();
break;
default: return parent::getName();
}
}
/**
* Replaces all relative URIs with absolute ones
*
* @param string $content The source string
* @return string Returns the source string with all relative URIs replaced
* by absolute ones.
*/
private function replaceRelativeURI($content){
return preg_replace('/href="(?!http)/', 'href="' . self::URI . '/', $content);
}
/**
* Adds styles as attributes to tags with known classes
*
* @param object $html A simplehtmldom object
* @return object Returns the original object with styles added as
* attributes.
*/
private function inlineStyles($html){
foreach($html->find('.bz_obsolete') as $element) {
$element->style = 'text-decoration:line-through;';
}
return $html;
}
}

View File

@@ -3,9 +3,9 @@ require_once('MoebooruBridge.php');
class KonachanBridge extends MoebooruBridge {
const MAINTAINER = "mitsukarenai";
const NAME = "Konachan";
const URI = "http://konachan.com/";
const DESCRIPTION = "Returns images from given page";
const MAINTAINER = 'mitsukarenai';
const NAME = 'Konachan';
const URI = 'http://konachan.com/';
const DESCRIPTION = 'Returns images from given page';
}

View File

@@ -1,12 +1,12 @@
<?php
class KoreusBridge extends FeedExpander {
const MAINTAINER = "pit-fgfjiudghdf";
const NAME = "Koreus";
const URI = "http://www.koreus.com/";
const DESCRIPTION = "Returns the newest posts from Koreus (full text)";
const MAINTAINER = 'pit-fgfjiudghdf';
const NAME = 'Koreus';
const URI = 'http://www.koreus.com/';
const DESCRIPTION = 'Returns the newest posts from Koreus (full text)';
protected function parseItem($item) {
protected function parseItem($item){
$item = parent::parseItem($item);
$html = getSimpleHTMLDOMCached($item['uri']);

View File

@@ -1,78 +1,87 @@
<?php
class KununuBridge extends BridgeAbstract {
const MAINTAINER = "logmanoriginal";
const NAME = "Kununu Bridge";
const URI = "https://www.kununu.com/";
const MAINTAINER = 'logmanoriginal';
const NAME = 'Kununu Bridge';
const URI = 'https://www.kununu.com/';
const CACHE_TIMEOUT = 86400; // 24h
const DESCRIPTION = "Returns the latest reviews for a company and site of your choice.";
const DESCRIPTION = 'Returns the latest reviews for a company and site of your choice.';
const PARAMETERS = array(
'global' => array(
'site'=>array(
'name'=>'Site',
'type'=>'list',
'required'=>true,
'title'=>'Select your site',
'values'=>array(
'Austria'=>'at',
'Germany'=>'de',
'Switzerland'=>'ch',
'United States'=>'us'
)
),
'full'=>array(
'name'=>'Load full article',
'type'=>'checkbox',
'required'=>false,
'exampleValue'=>'checked',
'title'=>'Activate to load full article'
)
),
const PARAMETERS = array(
'global' => array(
'site' => array(
'name' => 'Site',
'type' => 'list',
'required' => true,
'title' => 'Select your site',
'values' => array(
'Austria' => 'at',
'Germany' => 'de',
'Switzerland' => 'ch',
'United States' => 'us'
)
),
'full' => array(
'name' => 'Load full article',
'type' => 'checkbox',
'required' => false,
'exampleValue' => 'checked',
'title' => 'Activate to load full article'
)
),
array(
'company' => array(
'name' => 'Company',
'required' => true,
'exampleValue' => 'kununu-us',
'title' => 'Insert company name (i.e. Kununu US) or URI path (i.e. kununu-us)'
)
)
);
array(
'company'=>array(
'name'=>'Company',
'required'=>true,
'exampleValue'=>'kununu-us',
'title'=>'Insert company name (i.e. Kununu US) or URI path (i.e. kununu-us)'
)
)
);
private $companyName = '';
private $companyName='';
public function getURI(){
if(!is_null($this->getInput('company')) && !is_null($this->getInput('site'))) {
public function getURI(){
$company = $this->encode_umlauts(strtolower(str_replace(' ', '-', trim($this->getInput('company')))));
$site=$this->getInput('site');
$section = '';
switch($site){
case 'at':
case 'de':
case 'ch':
$section = 'kommentare';
break;
case 'us':
$section = 'reviews';
break;
}
$company = $this->fixCompanyName($this->getInput('company'));
$site = $this->getInput('site');
$section = '';
return self::URI.$site.'/'.$company.'/'.$section;
}
switch($site) {
case 'at':
case 'de':
case 'ch':
$section = 'kommentare';
break;
case 'us':
$section = 'reviews';
break;
}
function getName(){
$company = $this->encode_umlauts(strtolower(str_replace(' ', '-', trim($this->getInput('company')))));
return ($this->companyName?:$company).' - '.self::NAME;
}
return self::URI . $site . '/' . $company . '/' . $section . '?sort=update_time_desc';
}
return parent::getURI();
}
function getName(){
if(!is_null($this->getInput('company'))) {
$company = $this->fixCompanyName($this->getInput('company'));
return ($this->companyName ?: $company) . ' - ' . self::NAME;
}
return parent::getName();
}
public function collectData(){
$full = $this->getInput('full');
$full = $this->getInput('full');
// Load page
$html = getSimpleHTMLDOM($this->getURI());
$html = getSimpleHTMLDOMCached($this->getURI());
if(!$html)
returnServerError('Unable to receive data from ' . $this->getURI() . '!');
// Update name for this request
$this->companyName = $this->extract_company_name($html);
$this->companyName = $this->extractCompanyName($html);
// Find the section with all the panels (reviews)
$section = $html->find('section.kununu-scroll-element', 0);
@@ -85,18 +94,21 @@ class KununuBridge extends BridgeAbstract {
returnServerError('Unable to find articles!');
// Go through all articles
foreach($articles as $article){
foreach($articles as $article) {
$item = array();
$item['author'] = $this->extract_article_author_position($article);
$item['timestamp'] = $this->extract_article_date($article);
$item['title'] = $this->extract_article_rating($article) . ' : ' . $this->extract_article_summary($article);
$item['uri'] = $this->extract_article_uri($article);
$item['author'] = $this->extractArticleAuthorPosition($article);
$item['timestamp'] = $this->extractArticleDate($article);
$item['title'] = $this->extractArticleRating($article)
. ' : '
. $this->extractArticleSummary($article);
$item['uri'] = $this->extractArticleUri($article);
if($full)
$item['content'] = $this->extract_full_description($item['uri']);
$item['content'] = $this->extractFullDescription($item['uri']);
else
$item['content'] = $this->extract_article_description($article);
$item['content'] = $this->extractArticleDescription($article);
$this->items[] = $item;
}
@@ -105,16 +117,26 @@ class KununuBridge extends BridgeAbstract {
/**
* Fixes relative URLs in the given text
*/
private function fix_url($text){
private function fixUrl($text){
return preg_replace('/href=(\'|\")\//i', 'href="'.self::URI, $text);
}
/*
* Returns a fixed version of the provided company name
*/
private function fixCompanyName($company){
$company = trim($company);
$company = str_replace(' ', '-', $company);
$company = strtolower($company);
return $this->encodeUmlauts($company);
}
/**
* Encodes unmlauts in the given text
*/
private function encode_umlauts($text){
$umlauts = Array("/ä/","/ö/","/ü/","/Ä/","/Ö/","/Ü/","/ß/");
$replace = Array("ae","oe","ue","Ae","Oe","Ue","ss");
private function encodeUmlauts($text){
$umlauts = Array('/ä/','/ö/','/ü/','/Ä/','/Ö/','/Ü/','/ß/');
$replace = Array('ae','oe','ue','Ae','Oe','Ue','ss');
return preg_replace($umlauts, $replace, $text);
}
@@ -122,13 +144,9 @@ class KununuBridge extends BridgeAbstract {
/**
* Returns the company name from the review html
*/
private function extract_company_name($html){
$panel = $html->find('div.panel', 0);
if($panel === false)
returnServerError('Cannot find panel for company name!');
$company_name = $panel->find('h1', 0);
if($company_name === false)
private function extractCompanyName($html){
$company_name = $html->find('h1[itemprop=name]', 0);
if(is_null($company_name))
returnServerError('Cannot find company name!');
return $company_name->plaintext;
@@ -137,21 +155,21 @@ class KununuBridge extends BridgeAbstract {
/**
* Returns the date from a given article
*/
private function extract_article_date($article){
private function extractArticleDate($article){
// They conviniently provide a time attribute for us :)
$date = $article->find('time[itemprop=dtreviewed]', 0);
if($date === false)
$date = $article->find('meta[itemprop=dateCreated]', 0);
if(is_null($date))
returnServerError('Cannot find article date!');
return strtotime($date->datetime);
return strtotime($date->content);
}
/**
* Returns the rating from a given article
*/
private function extract_article_rating($article){
private function extractArticleRating($article){
$rating = $article->find('span.rating', 0);
if($rating === false)
if(is_null($rating))
returnServerError('Cannot find article rating!');
return $rating->getAttribute('aria-label');
@@ -160,9 +178,9 @@ class KununuBridge extends BridgeAbstract {
/**
* Returns the summary from a given article
*/
private function extract_article_summary($article){
$summary = $article->find('[itemprop=summary]', 0);
if($summary === false)
private function extractArticleSummary($article){
$summary = $article->find('[itemprop=name]', 0);
if(is_null($summary))
returnServerError('Cannot find article summary!');
return strip_tags($summary->innertext);
@@ -171,14 +189,9 @@ class KununuBridge extends BridgeAbstract {
/**
* Returns the URI from a given article
*/
private function extract_article_uri($article){
// Notice: This first part is the same as in extract_article_summary!
$summary = $article->find('[itemprop=summary]', 0);
if($summary === false)
returnServerError('Cannot find article summary!');
$anchor = $summary->find('a', 0);
if($anchor === false)
private function extractArticleUri($article){
$anchor = $article->find('h1.review-title a', 0);
if(is_null($anchor))
returnServerError('Cannot find article URI!');
return self::URI . $anchor->href;
@@ -187,17 +200,17 @@ class KununuBridge extends BridgeAbstract {
/**
* Returns the position of the author from a given article
*/
private function extract_article_author_position($article){
// We need to parse the aside manually
$aside = $article->find('aside', 0);
if($aside === false)
returnServerError('Cannot find article author information!');
private function extractArticleAuthorPosition($article){
// We need to parse the user-content manually
$user_content = $article->find('div.user-content', 0);
if(is_null($user_content))
returnServerError('Cannot find user content!');
// Go through all h2 elements to find index of required span (I know... it's stupid)
$author_position = 'Unknown';
foreach($aside->find('h2') as $subject){
if(stristr(strtolower($subject->plaintext), 'position')){ /* This works for at, ch, de, us */
$author_position = $subject->next_sibling()->plaintext;
foreach($user_content->find('div') as $content) {
if(stristr(strtolower($content->plaintext), 'position')) { /* This works for at, ch, de, us */
$author_position = $content->next_sibling()->plaintext;
break;
}
}
@@ -208,18 +221,18 @@ class KununuBridge extends BridgeAbstract {
/**
* Returns the description from a given article
*/
private function extract_article_description($article){
$description = $article->find('div[itemprop=description]', 0);
if($description === false)
private function extractArticleDescription($article){
$description = $article->find('[itemprop=reviewBody]', 0);
if(is_null($description))
returnServerError('Cannot find article description!');
return $this->fix_url($description->innertext);
return $this->fixUrl($description->innertext);
}
/**
* Returns the full description from a given uri
*/
private function extract_full_description($uri){
private function extractFullDescription($uri){
// Load full article
$html = getSimpleHTMLDOMCached($uri);
if($html === false)
@@ -227,10 +240,10 @@ class KununuBridge extends BridgeAbstract {
// Find the article
$article = $html->find('article', 0);
if($article === false)
if(is_null($article))
returnServerError('Cannot find article!');
// Luckily they use the same layout for the review overview and full article pages :)
return $this->extract_article_description($article);
return $this->extractArticleDescription($article);
}
}

View File

@@ -1,144 +1,265 @@
<?php
class LWNprevBridge extends BridgeAbstract{
const MAINTAINER = 'Pierre Mazière';
const NAME = 'LWN Free Weekly Edition';
const URI = 'https://lwn.net/';
const CACHE_TIMEOUT = 604800; // 1 week
const DESCRIPTION = 'LWN Free Weekly Edition available one week late';
const MAINTAINER = 'Pierre Mazière';
const NAME = 'LWN Free Weekly Edition';
const URI = 'https://lwn.net/';
const CACHE_TIMEOUT = 604800; // 1 week
const DESCRIPTION = 'LWN Free Weekly Edition available one week late';
function getURI(){
return self::URI.'free/bigpage';
}
private $editionTimeStamp;
private function jumpToNextTag(&$node){
while($node && $node->nodeType===XML_TEXT_NODE){
$nextNode=$node->nextSibling;
if(!$nextNode){
break;
}
$node=$nextNode;
}
}
function getURI(){
return self::URI.'free/bigpage';
}
private function jumpToPreviousTag(&$node){
while($node && $node->nodeType===XML_TEXT_NODE){
$previousNode=$node->previousSibling;
if(!$previousNode){
break;
}
$node=$previousNode;
}
}
private function jumpToNextTag(&$node){
while($node && $node->nodeType === XML_TEXT_NODE) {
$nextNode = $node->nextSibling;
if(!$nextNode) {
break;
}
$node = $nextNode;
}
}
public function collectData(){
// Because the LWN page is written in loose HTML and not XHTML,
// Simple HTML Dom is not accurate enough for the job
$content=getContents($this->getURI())
or returnServerError('No results for LWNprev');
private function jumpToPreviousTag(&$node){
while($node && $node->nodeType === XML_TEXT_NODE) {
$previousNode = $node->previousSibling;
if(!$previousNode) {
break;
}
$node = $previousNode;
}
}
libxml_use_internal_errors(true);
$html=new DOMDocument();
$html->loadHTML($content);
libxml_clear_errors();
public function collectData(){
// Because the LWN page is written in loose HTML and not XHTML,
// Simple HTML Dom is not accurate enough for the job
$content = getContents($this->getURI())
or returnServerError('No results for LWNprev');
$cat1='';
$cat2='';
$contents = explode('<b>Page editor</b>', $content);
foreach($html->getElementsByTagName('a') as $a){
if($a->textContent==='Multi-page format'){
break;
}
}
$realURI=self::URI.$a->getAttribute('href');
$URICounter=0;
foreach($contents as $content) {
if(strpos($content, '<html>') === false) {
$content = <<<EOD
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html><head><title>LWN</title></head><body>{$content}</body></html>
EOD;
} else {
$content = $content.'</body></html>';
}
$edition=$html->getElementsByTagName('h1')->item(0)->textContent;
$editionTimeStamp=strtotime(
substr($edition,strpos($edition,'for ')+strlen('for '))
);
libxml_use_internal_errors(true);
$html = new DOMDocument();
$html->loadHTML($content);
libxml_clear_errors();
foreach($html->getElementsByTagName('h2') as $h2){
if($h2->getAttribute('class')!=='SummaryHL'){
continue;
}
$edition = $html->getElementsByTagName('h1');
if($edition->length !== 0) {
$text = $edition->item(0)->textContent;
$this->editionTimeStamp = strtotime(
substr($text, strpos($text, 'for ') + strlen('for '))
);
}
$item = array();
if(strpos($content, 'Cat1HL') === false) {
$items = $this->getFeatureContents($html);
} elseif(strpos($content, 'Cat3HL') === false) {
$items = $this->getBriefItems($html);
} else {
$items = $this->getAnnouncements($html);
}
$h2NextSibling=$h2->nextSibling;
$this->jumpToNextTag($h2NextSibling);
$this->items = array_merge($this->items, $items);
}
}
switch($h2NextSibling->getAttribute('class')){
case 'FeatureByline':
$item['author']=$h2NextSibling->getElementsByTagName('b')->item(0)->textContent;
break;
case 'GAByline':
$text=$h2NextSibling->textContent;
$item['author']=substr($text,strpos($text,'by '));
break;
default:
$item['author']='LWN';
break;
};
private function getArticleContent(&$title){
$link = $title->firstChild;
$this->jumpToNextTag($link);
$item['uri'] = self::URI;
if($link->nodeName === 'a') {
$item['uri'] .= $link->getAttribute('href');
}
$h2FirstChild=$h2->firstChild;
$this->jumpToNextTag($h2FirstChild);
if($h2FirstChild->nodeName==='a'){
$item['uri']=self::URI.$h2FirstChild->getAttribute('href');
}else{
$item['uri']=$realURI.'#'.$URICounter;
}
$URICounter++;
$item['timestamp'] = $this->editionTimeStamp;
$item['timestamp']=$editionTimeStamp+$URICounter;
$node = $title;
$content = '';
$contentEnd = false;
while(!$contentEnd) {
$node = $node->nextSibling;
if(!$node || (
$node->nodeType !== XML_TEXT_NODE &&
$node->nodeName === 'h2' || (
!is_null($node->attributes) &&
!is_null($class = $node->attributes->getNamedItem('class')) &&
in_array($class->nodeValue, array('Cat1HL','Cat2HL'))
)
)
) {
$contentEnd = true;
} else {
$content .= $node->C14N();
}
}
$item['content'] = $content;
return $item;
}
$h2PrevSibling=$h2->previousSibling;
$this->jumpToPreviousTag($h2PrevSibling);
switch($h2PrevSibling->getAttribute('class')){
case 'Cat2HL':
$cat2=$h2PrevSibling->textContent;
$h2PrevSibling=$h2PrevSibling->previousSibling;
$this->jumpToPreviousTag($h2PrevSibling);
if($h2PrevSibling->getAttribute('class')!=='Cat1HL'){
break;
}
$cat1=$h2PrevSibling->textContent;
break;
case 'Cat1HL':
$cat1=$h2PrevSibling->textContent;
$cat2='';
break;
default:
break;
}
$h2PrevSibling=null;
private function getFeatureContents(&$html){
$items = array();
foreach($html->getElementsByTagName('h2') as $title) {
if($title->getAttribute('class') !== 'SummaryHL') {
continue;
}
$item['title']='';
if(!empty($cat1)){
$item['title'].='['.$cat1.($cat2?'/'.$cat2:'').'] ';
}
$item['title'].=$h2->textContent;
$item = array();
$node=$h2;
$content='';
$contentEnd=false;
while(!$contentEnd){
$node=$node->nextSibling;
if(
!$node || (
$node->nodeType!==XML_TEXT_NODE && (
$node->nodeName==='h2' ||
(!is_null($node->attributes) && !is_null($class=$node->attributes->getNamedItem('class')) &&
in_array($class->nodeValue,array('Cat1HL','Cat2HL')))
)
)
){
$contentEnd=true;
}else{
$content.=$node->C14N();
}
}
$item['content']=$content;
$this->items[]=$item;
}
}
$author = $title->nextSibling;
$this->jumpToNextTag($author);
if($author->getAttribute('class') === 'FeatureByline') {
$item['author'] = $author->getElementsByTagName('b')->item(0)->textContent;
} else {
continue;
}
$item['title'] = $title->textContent;
$items[] = array_merge($item, $this->getArticleContent($title));
}
return $items;
}
private function getItemPrefix(&$cat, &$cats){
$cat1 = '';
$cat2 = '';
$cat3 = '';
switch($cat->getAttribute('class')) {
case 'Cat3HL':
$cat3 = $cat->textContent;
$cat = $cat->previousSibling;
$this->jumpToPreviousTag($cat);
$cats[2] = $cat3;
if($cat->getAttribute('class') !== 'Cat2HL') {
break;
}
case 'Cat2HL':
$cat2 = $cat->textContent;
$cat = $cat->previousSibling;
$this->jumpToPreviousTag($cat);
$cats[1] = $cat2;
if(empty($cat3)) {
$cats[2] = '';
}
if($cat->getAttribute('class') !== 'Cat1HL') {
break;
}
case 'Cat1HL':
$cat1 = $cat->textContent;
$cats[0] = $cat1;
if(empty($cat3)) {
$cats[2] = '';
}
if(empty($cat2)) {
$cats[1] = '';
}
break;
default:
break;
}
$prefix = '';
if(!empty($cats[0])) {
$prefix .= '['.$cats[0].($cats[1] ? '/'.$cats[1] : '').'] ';
}
return $prefix;
}
private function getAnnouncements(&$html){
$items = array();
$cats = array('','','');
foreach($html->getElementsByTagName('p') as $newsletters) {
if($newsletters->getAttribute('class') !== 'Cat3HL') {
continue;
}
$item = array();
$item['uri'] = self::URI.'#'.count($items);
$item['timestamp'] = $this->editionTimeStamp;
$item['author'] = 'LWN';
$cat = $newsletters->previousSibling;
$this->jumpToPreviousTag($cat);
$prefix = $this->getItemPrefix($cat, $cats);
$item['title'] = $prefix.' '.$newsletters->textContent;
$node = $newsletters;
$content = '';
$contentEnd = false;
while(!$contentEnd) {
$node = $node->nextSibling;
if(!$node || (
$node->nodeType !== XML_TEXT_NODE && (
!is_null($node->attributes) &&
!is_null($class = $node->attributes->getNamedItem('class')) &&
in_array($class->nodeValue, array('Cat1HL','Cat2HL','Cat3HL'))
)
)
) {
$contentEnd = true;
} else {
$content .= $node->C14N();
}
}
$item['content'] = $content;
$items[] = $item;
}
foreach($html->getElementsByTagName('h2') as $title) {
if($title->getAttribute('class') !== 'SummaryHL') {
continue;
}
$item = array();
$cat = $title->previousSibling;
$this->jumpToPreviousTag($cat);
$cat = $cat->previousSibling;
$this->jumpToPreviousTag($cat);
$prefix = $this->getItemPrefix($cat, $cats);
$item['title'] = $prefix.' '.$title->textContent;
$items[] = array_merge($item, $this->getArticleContent($title));
}
return $items;
}
private function getBriefItems(&$html){
$items = array();
$cats = array('','','');
foreach($html->getElementsByTagName('h2') as $title) {
if($title->getAttribute('class') !== 'SummaryHL') {
continue;
}
$item = array();
$cat = $title->previousSibling;
$this->jumpToPreviousTag($cat);
$cat = $cat->previousSibling;
$this->jumpToPreviousTag($cat);
$prefix = $this->getItemPrefix($cat, $cats);
$item['title'] = $prefix.' '.$title->textContent;
$items[] = array_merge($item, $this->getArticleContent($title));
}
return $items;
}
}
?>

362
bridges/LeBonCoinBridge.php Executable file → Normal file
View File

@@ -1,185 +1,221 @@
<?php
class LeBonCoinBridge extends BridgeAbstract{
class LeBonCoinBridge extends BridgeAbstract {
const MAINTAINER = "16mhz";
const NAME = "LeBonCoin";
const URI = "http://www.leboncoin.fr/";
const DESCRIPTION = "Returns most recent results from LeBonCoin for a region, and optionally a category and a keyword .";
const MAINTAINER = 'jacknumber';
const NAME = 'LeBonCoin';
const URI = 'https://www.leboncoin.fr/';
const DESCRIPTION = 'Returns most recent results from LeBonCoin';
const PARAMETERS = array( array(
'k'=>array('name'=>'Mot Clé'),
'r'=>array(
'name'=>'Région',
'type'=>'list',
'values'=>array(
'Toute la France'=>'ile_de_france/occasions',
'Alsace'=>'alsace',
'Aquitaine'=>'aquitaine',
'Auvergne'=>'auvergne',
'Basse Normandie'=>'basse_normandie',
'Bourgogne'=>'bourgogne',
'Bretagne'=>'bretagne',
'Centre'=>'centre',
'Champagne Ardenne'=>'champagne_ardenne',
'Corse'=>'corse',
'Franche Comté'=>'franche_comte',
'Haute Normandie'=>'haute_normandie',
'Ile de France'=>'ile_de_france',
'Languedoc Roussillon'=>'languedoc_roussillon',
'Limousin'=>'limousin',
'Lorraine'=>'lorraine',
'Midi Pyrénées'=>'midi_pyrenees',
'Nord Pas De Calais'=>'nord_pas_de_calais',
'Pays de la Loire'=>'pays_de_la_loire',
'Picardie'=>'picardie',
'Poitou Charentes'=>'poitou_charentes',
'Provence Alpes Côte d\'Azur'=>'provence_alpes_cote_d_azur',
'Rhône-Alpes'=>'rhone_alpes',
'Guadeloupe'=>'guadeloupe',
'Martinique'=>'martinique',
'Guyane'=>'guyane',
'Réunion'=>'reunion'
)
),
'c'=>array(
'name'=>'Catégorie',
'type'=>'list',
'values'=>array(
'TOUS'=>'',
'EMPLOI'=>'_emploi_',
'VEHICULES'=>array(
'Tous'=>'_vehicules_',
'Voitures'=>'voitures',
'Motos'=>'motos',
'Caravaning'=>'caravaning',
'Utilitaires'=>'utilitaires',
'Équipement Auto'=>'equipement_auto',
'Équipement Moto'=>'equipement_moto',
'Équipement Caravaning'=>'equipement_caravaning',
'Nautisme'=>'nautisme',
'Équipement Nautisme'=>'equipement_nautisme'
),
'IMMOBILIER'=>array(
'Tous'=>'_immobilier_',
'Ventes immobilières'=>'ventes_immobilieres',
'Locations'=>'locations',
'Colocations'=>'colocations',
'Bureaux & Commerces'=>'bureaux_commerces'
),
'VACANCES'=>array(
'Tous'=>'_vacances_',
'Location gîtes'=>'locations_gites',
'Chambres d\'hôtes'=>'chambres_d_hotes',
'Campings'=>'campings',
'Hôtels'=>'hotels',
'Hébergements insolites'=>'hebergements_insolites'
),
'MULTIMEDIA'=>array(
'Tous'=>'_multimedia_',
'Informatique'=>'informatique',
'Consoles & Jeux vidéo'=>'consoles_jeux_video',
'Image & Son'=>'image_son',
'Téléphonie'=>'telephonie'
),
'LOISIRS'=>array(
'Tous'=>'_loisirs_',
'DVD / Films'=>'dvd_films',
'CD / Musique'=>'cd_musique',
'Livres'=>'livres',
'Animaux'=>'animaux',
'Vélos'=>'velos',
'Sports & Hobbies'=>'sports_hobbies',
'Instruments de musique'=>'instruments_de_musique',
'Collection'=>'collection',
'Jeux & Jouets'=>'jeux_jouets',
'Vins & Gastronomie'=>'vins_gastronomie'
),
'MATÉRIEL PROFESSIONNEL'=>array(
'Tous'=>'_materiel_professionnel_',
'Matériel Agricole'=>'mateiel_agricole',
'Transport - Manutention'=>'transport_manutention',
'BTP - Chantier - Gros-œuvre'=>'btp_chantier_gros_oeuvre',
'Outillage - Matériaux 2nd-œuvre'=>'outillage_materiaux_2nd_oeuvre',
'Équipements Industriels'=>'equipement_industriels',
'Restauration - Hôtellerie'=>'restauration_hotellerie',
'Fournitures de Bureau'=>'fournitures_de_bureau',
'Commerces & Marchés'=>'commerces_marches',
'Matériel médical'=>'materiel_medical'
),
'SERVICES'=>array(
'Tous'=>'_services_',
'Prestations de services'=>'prestations_de_services',
'Billetterie'=>'billetterie',
'Évènements'=>'evenements',
'Cours particuliers'=>'cours_particuliers',
'Covoiturage'=>'covoiturage'
),
'MAISON'=>array(
'Tous'=>'_maison_',
'Ameublement'=>'ameublement',
'Électroménager'=>'electromenager',
'Arts de la table'=>'arts_de_la_table',
'Décoration'=>'decoration',
'Linge de maison'=>'linge_de_maison',
'Bricolage'=>'bricolage',
'Jardinage'=>'jardinage',
'Vêtements'=>'vetements',
'Chaussures'=>'chaussures',
'Accessoires & Bagagerie'=>'accessoires_bagagerie',
'Montres & Bijoux'=>'montres_bijoux',
'Équipement bébé'=>'equipement_bebe',
'Vêtements bébé'=>'vetements_bebe'
),
'AUTRES'=>'autres'
)
)
)
);
const PARAMETERS = array(
array(
'k' => array('name' => 'Mot Clé'),
'r' => array(
'name' => 'Région',
'type' => 'list',
'values' => array(
'Toute la France' => '',
'Alsace' => '1',
'Aquitaine' => '2',
'Auvergne' => '3',
'Basse Normandie' => '4',
'Bourgogne' => '5',
'Bretagne' => '6',
'Centre' => '7',
'Champagne Ardenne' => '8',
'Corse' => '9',
'Franche Comté' => '10',
'Haute Normandie' => '11',
'Ile de France' => '12',
'Languedoc Roussillon' => '13',
'Limousin' => '14',
'Lorraine' => '15',
'Midi Pyrénées' => '16',
'Nord Pas De Calais' => '17',
'Pays de la Loire' => '18',
'Picardie' => '19',
'Poitou Charentes' => '20',
'Provence Alpes Côte d\'Azur' => '21',
'Rhône-Alpes' => '22',
'Guadeloupe' => '23',
'Martinique' => '24',
'Guyane' => '25',
'Réunion' => '26'
)
),
'c' => array(
'name' => 'Catégorie',
'type' => 'list',
'values' => array(
'Toutes catégories' => '',
'EMPLOI' => array(
'Emploi et recrutement' => '71',
'Offres d\'emploi et jobs' => '33'
),
'VEHICULES' => array(
'Tous' => '1',
'Voitures' => '2',
'Motos' => '3',
'Caravaning' => '4',
'Utilitaires' => '5',
'Equipement Auto' => '6',
'Equipement Moto' => '44',
'Equipement Caravaning' => '50',
'Nautisme' => '7',
'Equipement Nautisme' => '51'
),
'IMMOBILIER' => array(
'Tous' => '8',
'Ventes immobilières' => '9',
'Locations' => '10',
'Colocations' => '11',
'Bureaux & Commerces' => '13'
),
'VACANCES' => array(
'Tous' => '66',
'Locations & Gîtes' => '12',
'Chambres d\'hôtes' => '67',
'Campings' => '68',
'Hôtels' => '69',
'Hébergements insolites' => '70'
),
'MULTIMEDIA' => array(
'Tous' => '14',
'Informatique' => '15',
'Consoles & Jeux vidéo' => '43',
'Image & Son' => '16',
'Téléphonie' => '17'
),
'LOISIRS' => array(
'Tous' => '24',
'DVD / Films' => '25',
'CD / Musique' => '26',
'Livres' => '27',
'Animaux' => '28',
'Vélos' => '55',
'Sports & Hobbies' => '29',
'Instruments de musique' => '30',
'Collection' => '40',
'Jeux & Jouets' => '41',
'Vins & Gastronomie' => '48'
),
'MATERIEL PROFESSIONNEL' => array(
'Tous' => '56',
'Matériel Agricole' => '57',
'Transport - Manutention' => '58',
'BTP - Chantier Gros-oeuvre' => '59',
'Outillage - Matériaux 2nd-oeuvre' => '60',
'Équipements Industriels' => '32',
'Restauration - Hôtellerie' => '61',
'Fournitures de Bureau' => '62',
'Commerces & Marchés' => '63',
'Matériel Médical' => '64'
),
'SERVICES' => array(
'Tous' => '31',
'Prestations de services' => '34',
'Billetterie' => '35',
'Evénements' => '49',
'Cours particuliers' => '36',
'Covoiturage' => '65'
),
'MAISON' => array(
'Tous' => '18',
'Ameublement' => '19',
'Electroménager' => '20',
'Arts de la table' => '45',
'Décoration' => '39',
'Linge de maison' => '46',
'Bricolage' => '21',
'Jardinage' => '52',
'Vêtements' => '22',
'Chaussures' => '53',
'Accessoires & Bagagerie' => '47',
'Montres & Bijoux' => '42',
'Equipement bébé' => '23',
'Vêtements bébé' => '54',
),
'AUTRES' => '37'
)
),
'o' => array(
'name' => 'Vendeur',
'type' => 'list',
'values' => array(
'Tous' => '',
'Particuliers' => 'private',
'Professionnels' => 'pro',
)
)
)
);
public function collectData(){
$category=$this->getInput('c');
if (empty($category)){
$category='annonces';
}
$params = array(
'text' => $this->getInput('k'),
'region' => $this->getInput('r'),
'category' => $this->getInput('c'),
'owner_type' => $this->getInput('o'),
);
$html = getSimpleHTMLDOM(
self::URI.$category.'/offres/' . $this->getInput('r') . '/?'
.'f=a&th=1&'
.'q=' . urlencode($this->getInput('k'))
) or returnServerError('Could not request LeBonCoin.');
$url = self::URI . 'recherche/?' . http_build_query($params);
$html = getContents($url)
or returnServerError('Could not request LeBonCoin. Tried: ' . $url);
$list = $html->find('.tabsContent', 0);
if($list === NULL) {
if(!preg_match('/^<script>window.FLUX_STATE[^\r\n]*/m', $html, $matches)) {
returnServerError('Could not parse JSON in page content.');
}
$clean_match = str_replace(
array('</script>', '<script>window.FLUX_STATE = '),
array('', ''),
$matches[0]
);
$json = json_decode($clean_match);
if($json->adSearch->data->total === 0) {
return;
}
$tags = $list->find('li');
foreach($json->adSearch->data->ads as $element) {
foreach($tags as $element) {
$item['title'] = $element->subject;
$item['content'] = $element->body;
$item['date'] = $element->index_date;
$item['timestamp'] = strtotime($element->index_date);
$item['uri'] = $element->url;
$item['ad_type'] = $element->ad_type;
$item['author'] = $element->owner->name;
$element = $element->find('a', 0);
if(isset($element->location->city)) {
$item = array();
$item['uri'] = $element->href;
$title = html_entity_decode($element->getAttribute('title'));
$content_image = $element->find('div.item_image', 0)->find('.lazyload', 0);
$item['city'] = $element->location->city;
$item['content'] .= ' -- ' . $element->location->city;
if($content_image !== NULL) {
$content = '<img src="' . $content_image->getAttribute('data-imgsrc') . '" alt="thumbnail">';
} else {
$content = "";
}
$date = $element->find('aside.item_absolute', 0)->find('p.item_sup', 0);
$detailsList = $element->find('section.item_infos', 0);
if(isset($element->location->zipcode)) {
$item['zipcode'] = $element->location->zipcode;
}
for($i = 0; $i <= 1; $i++) $content .= $detailsList->find('p.item_supp', $i)->plaintext;
$price = $detailsList->find('h3.item_price', 0);
$content .= $price === NULL ? '' : $price->plaintext;
if(isset($element->price)) {
$item['price'] = $element->price[0];
$item['content'] .= ' -- ' . current($element->price) . '€';
}
if(isset($element->images->urls)) {
$item['thumbnail'] = $element->images->thumb_url;
$item['enclosures'] = array();
foreach($element->images->urls as $image) {
$item['enclosures'][] = $image;
}
}
$item['title'] = $title;
$item['content'] = $content . $date;
$this->items[] = $item;
}
}

View File

@@ -1,42 +1,44 @@
<?php
class LeMondeInformatiqueBridge extends FeedExpander {
const MAINTAINER = "ORelio";
const NAME = "Le Monde Informatique";
const URI = "http://www.lemondeinformatique.fr/";
const CACHE_TIMEOUT = 1800; // 30min
const DESCRIPTION = "Returns the newest articles.";
const MAINTAINER = 'ORelio';
const NAME = 'Le Monde Informatique';
const URI = 'http://www.lemondeinformatique.fr/';
const CACHE_TIMEOUT = 1800; // 30min
const DESCRIPTION = 'Returns the newest articles.';
public function collectData(){
$this->collectExpandableDatas(self::URI . 'rss/rss.xml', 10);
}
public function collectData(){
$this->collectExpandableDatas(self::URI . 'rss/rss.xml', 10);
}
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
$article_html = getSimpleHTMLDOMCached($item['uri'])
or returnServerError('Could not request LeMondeInformatique: ' . $item['uri']);
$item['content'] = $this->CleanArticle($article_html->find('div#article', 0)->innertext);
$item['title'] = $article_html->find('h1.cleanprint-title', 0)->plaintext;
return $item;
}
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
$article_html = getSimpleHTMLDOMCached($item['uri'])
or returnServerError('Could not request LeMondeInformatique: ' . $item['uri']);
$item['content'] = $this->cleanArticle($article_html->find('div#article', 0)->innertext);
$item['title'] = $article_html->find('h1.cleanprint-title', 0)->plaintext;
return $item;
}
function StripCDATA($string) {
$string = str_replace('<![CDATA[', '', $string);
$string = str_replace(']]>', '', $string);
return $string;
}
private function stripCDATA($string){
$string = str_replace('<![CDATA[', '', $string);
$string = str_replace(']]>', '', $string);
return $string;
}
function StripWithDelimiters($string, $start, $end) {
while (strpos($string, $start) !== false) {
$section_to_remove = substr($string, strpos($string, $start));
$section_to_remove = substr($section_to_remove, 0, strpos($section_to_remove, $end) + strlen($end));
$string = str_replace($section_to_remove, '', $string);
} return $string;
}
private function stripWithDelimiters($string, $start, $end){
while(strpos($string, $start) !== false) {
$section_to_remove = substr($string, strpos($string, $start));
$section_to_remove = substr($section_to_remove, 0, strpos($section_to_remove, $end) + strlen($end));
$string = str_replace($section_to_remove, '', $string);
}
function CleanArticle($article_html) {
$article_html = $this->StripWithDelimiters($article_html, '<script', '</script>');
$article_html = $this->StripWithDelimiters($article_html, '<h1 class="cleanprint-title"', '</h1>');
return $article_html;
}
return $string;
}
private function cleanArticle($article_html){
$article_html = $this->stripWithDelimiters($article_html, '<script', '</script>');
$article_html = $this->stripWithDelimiters($article_html, '<h1 class="cleanprint-title"', '</h1>');
return $article_html;
}
}

View File

@@ -1,70 +1,68 @@
<?php
class LegifranceJOBridge extends BridgeAbstract{
class LegifranceJOBridge extends BridgeAbstract {
const MAINTAINER = 'Pierre Mazière';
const NAME = 'Journal Officiel de la République Française';
const URI = 'https://www.legifrance.gouv.fr/affichJO.do';
const DESCRIPTION = 'Returns the laws and decrees officially registered daily in France';
const MAINTAINER = 'Pierre Mazière';
const NAME = 'Journal Officiel de la République Française';
const URI = 'https://www.legifrance.gouv.fr/affichJO.do';
const DESCRIPTION = 'Returns the laws and decrees officially registered daily in France';
const PARAMETERS=array();
const PARAMETERS = array();
private $author;
private $timestamp;
private $uri;
private $author;
private $timestamp;
private $uri;
private function extractItem($section,$subsection=null,$origin=null){
$item=array();
$item['author']=$this->author;
$item['timestamp']=$this->timestamp;
$item['uri']=$this->uri.'#'.count($this->items);
$item['title']=$section->plaintext;
private function extractItem($section, $subsection = null, $origin = null){
$item = array();
$item['author'] = $this->author;
$item['timestamp'] = $this->timestamp;
$item['uri'] = $this->uri . '#' . count($this->items);
$item['title'] = $section->plaintext;
if(!is_null($origin)){
$item['title']='[ '.$item['title'].' / '.$subsection->plaintext.' ] '.$origin->plaintext;
$data=$origin;
}elseif(!is_null($subsection)){
$item['title']='[ '.$item['title'].' ] '.$subsection->plaintext;
$data=$subsection;
}else{
$data=$section;
}
if(!is_null($origin)) {
$item['title'] = '[ ' . $item['title'] . ' / ' . $subsection->plaintext . ' ] ' . $origin->plaintext;
$data = $origin;
} elseif(!is_null($subsection)) {
$item['title'] = '[ ' . $item['title'] . ' ] ' . $subsection->plaintext;
$data = $subsection;
} else {
$data = $section;
}
$item['content']='';
foreach($data->nextSibling()->find('a') as $content){
$text=$content->plaintext;
$href=$content->nextSibling()->getAttribute('resource');
$item['content'].='<p><a href="'.$href.'">'.$text.'</a></p>';
}
return $item;
}
$item['content'] = '';
foreach($data->nextSibling()->find('a') as $content) {
$text = $content->plaintext;
$href = $content->nextSibling()->getAttribute('resource');
$item['content'] .= '<p><a href="' . $href . '">' . $text . '</a></p>';
}
return $item;
}
public function collectData(){
$html=getSimpleHTMLDOM(self::URI)
or $this->returnServer('Unable to download '.self::URI);
public function collectData(){
$html = getSimpleHTMLDOM(self::URI)
or $this->returnServer('Unable to download ' . self::URI);
$this->author=trim($html->find('h2.title',0)->plaintext);
$uri=$html->find('h2.titleELI',0)->plaintext;
$this->uri=trim(substr($uri,strpos($uri,'https')));
$this->timestamp=strtotime(substr($this->uri,strpos($this->uri,'eli/jo/')+strlen('eli/jo/')));
$this->author = trim($html->find('h2.titleJO', 0)->plaintext);
$uri = $html->find('h2.titleELI', 0)->plaintext;
$this->uri = trim(substr($uri, strpos($uri, 'https')));
$this->timestamp = strtotime(substr($this->uri, strpos($this->uri, 'eli/jo/') + strlen('eli/jo/'), -5));
foreach($html->find('h3') as $section){
$subsections=$section->nextSibling()->find('h4');
foreach($subsections as $subsection){
$origins=$subsection->nextSibling()->find('h5');
foreach($origins as $origin){
$this->items[]=$this->extractItem($section,$subsection,$origin);
}
if(!empty($origins)){
continue;
}
$this->items[]=$this->extractItem($section,$subsection);
}
if(!empty($subsections)){
continue;
}
$this->items[]=$this->extractItem($section);
}
}
foreach($html->find('h3') as $section) {
$subsections = $section->nextSibling()->find('h4');
foreach($subsections as $subsection) {
$origins = $subsection->nextSibling()->find('h5');
foreach($origins as $origin) {
$this->items[] = $this->extractItem($section, $subsection, $origin);
}
if(!empty($origins)) {
continue;
}
$this->items[] = $this->extractItem($section, $subsection);
}
if(!empty($subsections)) {
continue;
}
$this->items[] = $this->extractItem($section);
}
}
}

View File

@@ -1,47 +1,45 @@
<?php
class LesJoiesDuCodeBridge extends BridgeAbstract{
class LesJoiesDuCodeBridge extends BridgeAbstract {
const MAINTAINER = "superbaillot.net";
const NAME = "Les Joies Du Code";
const URI = "http://lesjoiesducode.fr/";
const MAINTAINER = 'superbaillot.net';
const NAME = 'Les Joies Du Code';
const URI = 'http://lesjoiesducode.fr/';
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = "LesJoiesDuCode";
const DESCRIPTION = 'LesJoiesDuCode';
public function collectData(){
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not request LesJoiesDuCode.');
public function collectData(){
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not request LesJoiesDuCode.');
foreach($html->find('div.blog-post') as $element) {
$item = array();
$temp = $element->find('h1 a', 0);
$titre = html_entity_decode($temp->innertext);
$url = $temp->href;
foreach($html->find('div.blog-post') as $element) {
$item = array();
$temp = $element->find('h1 a', 0);
$titre = html_entity_decode($temp->innertext);
$url = $temp->href;
$temp = $element->find('div.blog-post-content', 0);
$temp = $element->find('div.blog-post-content', 0);
// retrieve .gif instead of static .jpg
$images = $temp->find('p img');
foreach($images as $image){
$img_src = str_replace(".jpg",".gif",$image->src);
$image->src = $img_src;
}
$content = $temp->innertext;
// retrieve .gif instead of static .jpg
$images = $temp->find('p img');
foreach($images as $image) {
$img_src = str_replace('.jpg', '.gif', $image->src);
$image->src = $img_src;
}
$content = $temp->innertext;
$auteur = $temp->find('i', 0);
$pos = strpos($auteur->innertext, "by");
$auteur = $temp->find('i', 0);
$pos = strpos($auteur->innertext, 'by');
if($pos > 0)
{
$auteur = trim(str_replace("*/", "", substr($auteur->innertext, ($pos + 2))));
$item['author'] = $auteur;
}
if($pos > 0) {
$auteur = trim(str_replace('*/', '', substr($auteur->innertext, ($pos + 2))));
$item['author'] = $auteur;
}
$item['content'] .= trim($content);
$item['uri'] = $url;
$item['title'] = trim($titre);
$item['content'] .= trim($content);
$item['uri'] = $url;
$item['title'] = trim($titre);
$this->items[] = $item;
}
}
$this->items[] = $item;
}
}
}

View File

@@ -1,31 +1,31 @@
<?php
class LichessBridge extends FeedExpander {
const MAINTAINER = 'AmauryCarrade';
const NAME = 'Lichess Blog';
const URI = 'http://fr.lichess.org/blog';
const DESCRIPTION = 'Returns the 5 newest posts from the Lichess blog (full text)';
const MAINTAINER = 'AmauryCarrade';
const NAME = 'Lichess Blog';
const URI = 'http://fr.lichess.org/blog';
const DESCRIPTION = 'Returns the 5 newest posts from the Lichess blog (full text)';
public function collectData(){
$this->collectExpandableDatas(self::URI . '.atom', 5);
}
public function collectData(){
$this->collectExpandableDatas(self::URI . '.atom', 5);
}
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
$item['content'] = $this->retrieve_lichess_post($item['uri']);
return $item;
}
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
$item['content'] = $this->retrieveLichessPost($item['uri']);
return $item;
}
private function retrieve_lichess_post($blog_post_uri){
$blog_post_html = getSimpleHTMLDOMCached($blog_post_uri);
$blog_post_div = $blog_post_html->find('#lichess_blog', 0);
private function retrieveLichessPost($blog_post_uri){
$blog_post_html = getSimpleHTMLDOMCached($blog_post_uri);
$blog_post_div = $blog_post_html->find('#lichess_blog', 0);
$post_chapo = $blog_post_div->find('.shortlede', 0)->innertext;
$post_content = $blog_post_div->find('.body', 0)->innertext;
$post_chapo = $blog_post_div->find('.shortlede', 0)->innertext;
$post_content = $blog_post_div->find('.body', 0)->innertext;
$content = '<p><em>' . $post_chapo . '</em></p>';
$content .= '<div>' . $post_content . '</div>';
$content = '<p><em>' . $post_chapo . '</em></p>';
$content .= '<div>' . $post_content . '</div>';
return $content;
}
return $content;
}
}

View File

@@ -1,36 +1,37 @@
<?php
class LinkedInCompanyBridge extends BridgeAbstract{
class LinkedInCompanyBridge extends BridgeAbstract {
const MAINTAINER = "regisenguehard";
const NAME = "LinkedIn Company";
const URI = "https://www.linkedin.com/";
const MAINTAINER = 'regisenguehard';
const NAME = 'LinkedIn Company';
const URI = 'https://www.linkedin.com/';
const CACHE_TIMEOUT = 21600; //6
const DESCRIPTION = "Returns most recent actus from Company on LinkedIn. (https://www.linkedin.com/company/<strong style=\"font-weight:bold;\">apple</strong>)";
const DESCRIPTION = 'Returns most recent actus from Company on LinkedIn.
(https://www.linkedin.com/company/<strong style=\"font-weight:bold;\">apple</strong>)';
const PARAMETERS = array( array(
'c'=>array(
'name'=>'Company name',
'required'=>true
)
));
const PARAMETERS = array( array(
'c' => array(
'name' => 'Company name',
'required' => true
)
));
public function collectData(){
$html = '';
$link = self::URI.'company/'.$this->getInput('c');
public function collectData(){
$html = '';
$link = self::URI . 'company/' . $this->getInput('c');
$html = getSimpleHTMLDOM($link)
or returnServerError('Could not request LinkedIn.');
$html = getSimpleHTMLDOM($link)
or returnServerError('Could not request LinkedIn.');
foreach($html->find('//*[@id="my-feed-post"]/li') as $element) {
$title = $element->find('span.share-body', 0)->innertext;
if ($title) {
$item = array();
$item['uri'] = $link;
$item['title'] = mb_substr(strip_tags($element->find('span.share-body', 0)->innertext), 0 ,100);
$item['content'] = strip_tags($element->find('span.share-body', 0)->innertext);
$this->items[] = $item;
$i++;
}
}
}
foreach($html->find('//*[@id="my-feed-post"]/li') as $element) {
$title = $element->find('span.share-body', 0)->innertext;
if($title) {
$item = array();
$item['uri'] = $link;
$item['title'] = mb_substr(strip_tags($element->find('span.share-body', 0)->innertext), 0, 100);
$item['content'] = strip_tags($element->find('span.share-body', 0)->innertext);
$this->items[] = $item;
$i++;
}
}
}
}

View File

@@ -1,11 +1,11 @@
<?php
require_once('MoebooruBridge.php');
class LolibooruBridge extends MoebooruBridge{
class LolibooruBridge extends MoebooruBridge {
const MAINTAINER = "mitsukarenai";
const NAME = "Lolibooru";
const URI = "https://lolibooru.moe/";
const DESCRIPTION = "Returns images from given page and tags";
const MAINTAINER = 'mitsukarenai';
const NAME = 'Lolibooru';
const URI = 'https://lolibooru.moe/';
const DESCRIPTION = 'Returns images from given page and tags';
}

View File

@@ -1,251 +1,249 @@
<?php
class MangareaderBridge extends BridgeAbstract {
const MAINTAINER = "logmanoriginal";
const NAME = "Mangareader Bridge";
const URI = "http://www.mangareader.net/";
const CACHE_TIMEOUT = 10800; // 3h
const DESCRIPTION = "Returns the latest updates, popular mangas or manga updates (new chapters)";
const MAINTAINER = 'logmanoriginal';
const NAME = 'Mangareader Bridge';
const URI = 'http://www.mangareader.net';
const CACHE_TIMEOUT = 10800; // 3h
const DESCRIPTION = 'Returns the latest updates, popular mangas or manga updates (new chapters)';
const PARAMETERS = array(
'Get latest updates' => array(),
'Get popular mangas' => array(
'category' => array(
'name' => 'Category',
'type' => 'list',
'required' => true,
'values' => array(
'All' => 'all',
'Action' => 'action',
'Adventure' => 'adventure',
'Comedy' => 'comedy',
'Demons' => 'demons',
'Drama' => 'drama',
'Ecchi' => 'ecchi',
'Fantasy' => 'fantasy',
'Gender Bender' => 'gender-bender',
'Harem' => 'harem',
'Historical' => 'historical',
'Horror' => 'horror',
'Josei' => 'josei',
'Magic' => 'magic',
'Martial Arts' => 'martial-arts',
'Mature' => 'mature',
'Mecha' => 'mecha',
'Military' => 'military',
'Mystery' => 'mystery',
'One Shot' => 'one-shot',
'Psychological' => 'psychological',
'Romance' => 'romance',
'School Life' => 'school-life',
'Sci-Fi' => 'sci-fi',
'Seinen' => 'seinen',
'Shoujo' => 'shoujo',
'Shoujoai' => 'shoujoai',
'Shounen' => 'shounen',
'Shounenai' => 'shounenai',
'Slice of Life' => 'slice-of-life',
'Smut' => 'smut',
'Sports' => 'sports',
'Super Power' => 'super-power',
'Supernatural' => 'supernatural',
'Tragedy' => 'tragedy',
'Vampire' => 'vampire',
'Yaoi' => 'yaoi',
'Yuri' => 'yuri'
),
'exampleValue' => 'All',
'title' => 'Select your category'
)
),
'Get manga updates' => array(
'path' => array(
'name' => 'Path',
'required' => true,
'pattern' => '[a-zA-Z0-9-_]*',
'exampleValue' => 'bleach, umi-no-kishidan',
'title' => 'URL part of desired manga'
),
'limit' => array(
'name' => 'Limit',
'type' => 'number',
'defaultValue' => 10,
'title' => 'Number of items to return [-1 returns all]'
)
)
);
const PARAMETERS = array(
'Get latest updates' => array(),
'Get popular mangas' => array(
'category' => array(
'name' => 'Category',
'type' => 'list',
'required' => true,
'values' => array(
'All' => 'all',
'Action' => 'action',
'Adventure' => 'adventure',
'Comedy' => 'comedy',
'Demons' => 'demons',
'Drama' => 'drama',
'Ecchi' => 'ecchi',
'Fantasy' => 'fantasy',
'Gender Bender' => 'gender-bender',
'Harem' => 'harem',
'Historical' => 'historical',
'Horror' => 'horror',
'Josei' => 'josei',
'Magic' => 'magic',
'Martial Arts' => 'martial-arts',
'Mature' => 'mature',
'Mecha' => 'mecha',
'Military' => 'military',
'Mystery' => 'mystery',
'One Shot' => 'one-shot',
'Psychological' => 'psychological',
'Romance' => 'romance',
'School Life' => 'school-life',
'Sci-Fi' => 'sci-fi',
'Seinen' => 'seinen',
'Shoujo' => 'shoujo',
'Shoujoai' => 'shoujoai',
'Shounen' => 'shounen',
'Shounenai' => 'shounenai',
'Slice of Life' => 'slice-of-life',
'Smut' => 'smut',
'Sports' => 'sports',
'Super Power' => 'super-power',
'Supernatural' => 'supernatural',
'Tragedy' => 'tragedy',
'Vampire' => 'vampire',
'Yaoi' => 'yaoi',
'Yuri' => 'yuri'
),
'exampleValue' => 'All',
'title' => 'Select your category'
)
),
'Get manga updates' => array(
'path' => array(
'name' => 'Path',
'required' => true,
'pattern' => '[a-zA-Z0-9-_]*',
'exampleValue' => 'bleach, umi-no-kishidan',
'title' => 'URL part of desired manga'
),
'limit' => array(
'name' => 'Limit',
'type' => 'number',
'defaultValue' => 10,
'title' => 'Number of items to return [-1 returns all]'
)
)
);
private $request = '';
private $request = '';
public function collectData(){
// We'll use the DOM parser for this as it makes navigation easier
$html = getContents($this->getURI());
if(!$html){
returnClientError('Could not receive data for ' . $path . '!');
}
libxml_use_internal_errors(true);
$doc = new DomDocument;
@$doc->loadHTML($html);
libxml_clear_errors();
public function collectData(){
// We'll use the DOM parser for this as it makes navigation easier
$html = getContents($this->getURI());
if(!$html) {
returnClientError('Could not receive data for ' . $path . '!');
}
libxml_use_internal_errors(true);
$doc = new DomDocument;
@$doc->loadHTML($html);
libxml_clear_errors();
// Navigate via XPath
$xpath = new DomXPath($doc);
// Navigate via XPath
$xpath = new DomXPath($doc);
$this->request = '';
switch($this->queriedContext){
case 'Get latest updates':
$this->request = 'Latest updates';
$this->get_latest_updates($xpath);
break;
case 'Get popular mangas':
// Find manga name within "Popular mangas for ..."
$pagetitle = $xpath->query(".//*[@id='bodyalt']/h1")->item(0)->nodeValue;
$this->request = substr($pagetitle, 0, strrpos($pagetitle, " -"));
$this->get_popular_mangas($xpath);
break;
case 'Get manga updates':
$limit = $this->getInput('limit');
if(empty($limit)){
$limit = self::PARAMETERS[$this->queriedContext]['limit']['defaultValue'];
}
$this->request = '';
switch($this->queriedContext) {
case 'Get latest updates':
$this->request = 'Latest updates';
$this->getLatestUpdates($xpath);
break;
case 'Get popular mangas':
// Find manga name within "Popular mangas for ..."
$pagetitle = $xpath->query(".//*[@id='bodyalt']/h1")->item(0)->nodeValue;
$this->request = substr($pagetitle, 0, strrpos($pagetitle, ' -'));
$this->getPopularMangas($xpath);
break;
case 'Get manga updates':
$limit = $this->getInput('limit');
if(empty($limit)) {
$limit = self::PARAMETERS[$this->queriedContext]['limit']['defaultValue'];
}
$this->request = $xpath->query(".//*[@id='mangaproperties']//*[@class='aname']")
->item(0)
->nodeValue;
$this->request = $xpath->query(".//*[@id='mangaproperties']//*[@class='aname']")
->item(0)
->nodeValue;
$this->get_manga_updates($xpath, $limit);
break;
}
$this->getMangaUpdates($xpath, $limit);
break;
}
// Return some dummy-data if no content available
if(empty($this->items)){
$item = array();
$item['content'] = "<p>No updates available</p>";
// Return some dummy-data if no content available
if(empty($this->items)) {
$item = array();
$item['content'] = '<p>No updates available</p>';
$this->items[] = $item;
}
}
$this->items[] = $item;
}
}
private function get_latest_updates($xpath){
// Query each item (consists of Manga + chapters)
$nodes = $xpath->query("//*[@id='latestchapters']/table//td");
private function getLatestUpdates($xpath){
// Query each item (consists of Manga + chapters)
$nodes = $xpath->query("//*[@id='latestchapters']/table//td");
foreach ($nodes as $node){
// Query the manga
$manga = $xpath->query("a[@class='chapter']", $node)->item(0);
foreach ($nodes as $node) {
// Query the manga
$manga = $xpath->query("a[@class='chapter']", $node)->item(0);
// Collect the chapters for each Manga
$chapters = $xpath->query("a[@class='chaptersrec']", $node);
// Collect the chapters for each Manga
$chapters = $xpath->query("a[@class='chaptersrec']", $node);
if (isset($manga) && $chapters->length >= 1){
$item = array();
$item['uri'] = self::URI . htmlspecialchars($manga->getAttribute('href'));
$item['title'] = htmlspecialchars($manga->nodeValue);
if (isset($manga) && $chapters->length >= 1) {
$item = array();
$item['uri'] = self::URI . htmlspecialchars($manga->getAttribute('href'));
$item['title'] = htmlspecialchars($manga->nodeValue);
// Add each chapter to the feed
$item['content'] = "";
// Add each chapter to the feed
$item['content'] = '';
foreach ($chapters as $chapter){
if($item['content'] <> ""){
$item['content'] .= "<br>";
}
$item['content'] .=
"<a href='"
. self::URI
. htmlspecialchars($chapter->getAttribute('href'))
. "'>"
. htmlspecialchars($chapter->nodeValue)
. "</a>";
}
foreach ($chapters as $chapter) {
if($item['content'] <> '') {
$item['content'] .= '<br>';
}
$item['content'] .= "<a href='"
. self::URI
. htmlspecialchars($chapter->getAttribute('href'))
. "'>"
. htmlspecialchars($chapter->nodeValue)
. '</a>';
}
$this->items[] = $item;
}
}
}
$this->items[] = $item;
}
}
}
private function get_popular_mangas($xpath){
// Query all mangas
$mangas = $xpath->query("//*[@id='mangaresults']/*[@class='mangaresultitem']");
private function getPopularMangas($xpath){
// Query all mangas
$mangas = $xpath->query("//*[@id='mangaresults']/*[@class='mangaresultitem']");
foreach ($mangas as $manga){
foreach ($mangas as $manga) {
// The thumbnail is encrypted in a css-style...
// format: "background-image:url('<the part which is actually interesting>')"
$mangaimgelement = $xpath->query(".//*[@class='imgsearchresults']", $manga)
->item(0)
->getAttribute('style');
$thumbnail = substr($mangaimgelement, 22, strlen($mangaimgelement) - 24);
// The thumbnail is encrypted in a css-style...
// format: "background-image:url('<the part which is actually interesting>')"
$mangaimgelement = $xpath->query(".//*[@class='imgsearchresults']", $manga)
->item(0)
->getAttribute('style');
$thumbnail = substr($mangaimgelement, 22, strlen($mangaimgelement) - 24);
$item = array();
$item['title'] = htmlspecialchars($xpath->query(".//*[@class='manga_name']//a", $manga)
->item(0)
->nodeValue);
$item['uri'] = self::URI . $xpath->query(".//*[@class='manga_name']//a", $manga)
->item(0)
->getAttribute('href');
$item['author'] = htmlspecialchars($xpath->query("//*[@class='author_name']", $manga)
->item(0)
->nodeValue);
$item['chaptercount'] = $xpath->query(".//*[@class='chapter_count']", $manga)
->item(0)
->nodeValue;
$item['genre'] = htmlspecialchars($xpath->query(".//*[@class='manga_genre']", $manga)
->item(0)
->nodeValue);
$item['content'] = <<<EOD
$item = array();
$item['title'] = htmlspecialchars($xpath->query(".//*[@class='manga_name']//a", $manga)
->item(0)
->nodeValue);
$item['uri'] = self::URI . $xpath->query(".//*[@class='manga_name']//a", $manga)
->item(0)
->getAttribute('href');
$item['author'] = htmlspecialchars($xpath->query("//*[@class='author_name']", $manga)
->item(0)
->nodeValue);
$item['chaptercount'] = $xpath->query(".//*[@class='chapter_count']", $manga)
->item(0)
->nodeValue;
$item['genre'] = htmlspecialchars($xpath->query(".//*[@class='manga_genre']", $manga)
->item(0)
->nodeValue);
$item['content'] = <<<EOD
<a href="{$item['uri']}"><img src="{$thumbnail}" alt="{$item['title']}" /></a>
<p>{$item['genre']}</p>
<p>{$item['chaptercount']}</p>
EOD;
$this->items[] = $item;
}
}
$this->items[] = $item;
}
}
private function get_manga_updates($xpath, $limit){
$query = "(.//*[@id='listing']//tr)[position() > 1]";
private function getMangaUpdates($xpath, $limit){
$query = "(.//*[@id='listing']//tr)[position() > 1]";
if($limit !== -1){
$query = "(.//*[@id='listing']//tr)[position() > 1][position() > last() - {$limit}]";
}
if($limit !== -1) {
$query = "(.//*[@id='listing']//tr)[position() > 1][position() > last() - {$limit}]";
}
$chapters = $xpath->query($query);
$chapters = $xpath->query($query);
foreach ($chapters as $chapter){
$item = array();
$item['title'] = htmlspecialchars($xpath->query("td[1]", $chapter)
->item(0)
->nodeValue);
$item['uri'] = self::URI . $xpath->query("td[1]/a", $chapter)
->item(0)
->getAttribute('href');
$item['timestamp'] = strtotime($xpath->query("td[2]", $chapter)
->item(0)
->nodeValue);
array_unshift($this->items, $item);
}
}
foreach ($chapters as $chapter) {
$item = array();
$item['title'] = htmlspecialchars($xpath->query('td[1]', $chapter)
->item(0)
->nodeValue);
$item['uri'] = self::URI . $xpath->query('td[1]/a', $chapter)
->item(0)
->getAttribute('href');
$item['timestamp'] = strtotime($xpath->query('td[2]', $chapter)
->item(0)
->nodeValue);
array_unshift($this->items, $item);
}
}
public function getURI(){
switch($this->queriedContext){
case 'Get latest updates':
$path = "latest";
break;
case 'Get popular mangas':
$path = "popular";
if($this->getInput('category') !== "all"){
$path .= "/" . $this->getInput('category');
}
break;
case 'Get manga updates':
$path = $this->getInput('path');
break;
}
return self::URI . $path;
}
public function getURI(){
switch($this->queriedContext) {
case 'Get latest updates':
$path = 'latest';
break;
case 'Get popular mangas':
$path = 'popular';
if($this->getInput('category') !== 'all') {
$path .= '/' . $this->getInput('category');
}
break;
case 'Get manga updates':
$path = $this->getInput('path');
break;
default: return parent::getURI();
}
return self::URI . '/' . $path;
}
public function getName(){
return (!empty($this->request) ? $this->request . ' - ' : '') . 'Mangareader Bridge';
}
public function getName(){
return (!empty($this->request) ? $this->request . ' - ' : '') . 'Mangareader Bridge';
}
}
?>

View File

@@ -1,11 +1,11 @@
<?php
require_once('Shimmie2Bridge.php');
class MilbooruBridge extends Shimmie2Bridge{
class MilbooruBridge extends Shimmie2Bridge {
const MAINTAINER = "mitsukarenai";
const NAME = "Milbooru";
const URI = "http://sheslostcontrol.net/moe/shimmie/";
const DESCRIPTION = "Returns images from given page";
const MAINTAINER = 'mitsukarenai';
const NAME = 'Milbooru';
const URI = 'http://sheslostcontrol.net/moe/shimmie/';
const DESCRIPTION = 'Returns images from given page';
}

View File

@@ -0,0 +1,53 @@
<?php
class MixCloudBridge extends BridgeAbstract {
const MAINTAINER = 'Alexis CHEMEL';
const NAME = 'MixCloud';
const URI = 'https://www.mixcloud.com';
const CACHE_TIMEOUT = 3600; // 1h
const DESCRIPTION = 'Returns latest musics on user stream';
const PARAMETERS = array(array(
'u' => array(
'name' => 'username',
'required' => true,
)
));
public function getName(){
if(!is_null($this->getInput('u'))) {
return 'MixCloud - ' . $this->getInput('u');
}
return parent::getName();
}
public function collectData(){
ini_set('user_agent', 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:53.0) Gecko/20100101 Firefox/53.0');
$html = getSimpleHTMLDOM(self::URI . '/' . $this->getInput('u'))
or returnServerError('Could not request MixCloud.');
foreach($html->find('section.card') as $element) {
$item = array();
$item['uri'] = self::URI . $element->find('hgroup.card-title h1 a', 0)->getAttribute('href');
$item['title'] = html_entity_decode(
$element->find('hgroup.card-title h1 a span', 0)->getAttribute('title'),
ENT_QUOTES
);
$image = $element->find('a.album-art img', 0);
if($image) {
$item['content'] = '<img src="' . $image->getAttribute('src') . '" />';
}
$item['author'] = trim($element->find('hgroup.card-title h2 a', 0)->innertext);
$this->items[] = $item;
}
}
}

View File

@@ -1,46 +1,56 @@
<?php
class MoebooruBridge extends BridgeAbstract{
class MoebooruBridge extends BridgeAbstract {
const NAME = "Moebooru";
const URI = "https://moe.dev.myconan.net/";
const CACHE_TIMEOUT = 1800; // 30min
const DESCRIPTION = "Returns images from given page";
const NAME = 'Moebooru';
const URI = 'https://moe.dev.myconan.net/';
const CACHE_TIMEOUT = 1800; // 30min
const DESCRIPTION = 'Returns images from given page';
const MAINTAINER = 'pmaziere';
const PARAMETERS = array( array(
'p'=>array(
'name'=>'page',
'defaultValue'=>1,
'type'=>'number'
),
't'=>array('name'=>'tags')
));
const PARAMETERS = array( array(
'p' => array(
'name' => 'page',
'defaultValue' => 1,
'type' => 'number'
),
't' => array(
'name' => 'tags'
)
));
protected function getFullURI(){
return $this->getURI().'post?'
.'page='.$this->getInput('p')
.'&tags='.urlencode($this->getInput('t'));
}
protected function getFullURI(){
return $this->getURI()
. 'post?page='
. $this->getInput('p')
. '&tags='
. urlencode($this->getInput('t'));
}
public function collectData(){
$html = getSimpleHTMLDOM($this->getFullURI())
or returnServerError('Could not request '.$this->getName());
public function collectData(){
$html = getSimpleHTMLDOM($this->getFullURI())
or returnServerError('Could not request ' . $this->getName());
$input_json = explode('Post.register(', $html);
foreach($input_json as $element)
$data[] = preg_replace('/}\)(.*)/', '}', $element);
unset($data[0]);
$input_json = explode('Post.register(', $html);
foreach($input_json as $element)
$data[] = preg_replace('/}\)(.*)/', '}', $element);
unset($data[0]);
foreach($data as $datai) {
$json = json_decode($datai, true);
$item = array();
$item['uri'] = $this->getURI() . '/post/show/' . $json['id'];
$item['postid'] = $json['id'];
$item['timestamp'] = $json['created_at'];
$item['imageUri'] = $json['file_url'];
$item['title'] = $this->getName() . ' | ' . $json['id'];
$item['content'] = '<a href="'
. $item['imageUri']
. '"><img src="'
. $json['preview_url']
. '" /></a><br>Tags: '
. $json['tags'];
foreach($data as $datai) {
$json = json_decode($datai, TRUE);
$item = array();
$item['uri'] = $this->getURI().'/post/show/'.$json['id'];
$item['postid'] = $json['id'];
$item['timestamp'] = $json['created_at'];
$item['imageUri'] = $json['file_url'];
$item['title'] = $this->getName().' | '.$json['id'];
$item['content'] = '<a href="' . $item['imageUri'] . '"><img src="' . $json['preview_url'] . '" /></a><br>Tags: '.$json['tags'];
$this->items[] = $item;
}
}
$this->items[] = $item;
}
}
}

Some files were not shown because too many files have changed in this diff Show More