1
0
mirror of https://github.com/RSS-Bridge/rss-bridge.git synced 2025-08-17 22:02:09 +02:00

Compare commits

..

82 Commits

Author SHA1 Message Date
logmanoriginal
c17b864242 [Configuration] Bump version to 2019-06-08 2019-06-08 20:04:57 +02:00
logmanoriginal
5ff3d0121c [README] Update list of contributors 2019-06-08 20:04:06 +02:00
Joseph
f00a054e0f [BrutBridge] Add new bridge (#1159) 2019-06-08 19:30:42 +02:00
logmanoriginal
5a9519967b [Exceptions] Add button to search for similar issues on GitHub
Users currently only get one option: to open a new issue on GitHub.
This can, however, result in duplicate issues, which is not desired.

This commit adds a second button to the error message, which links
to the GitHub issues tracker with the search query set to find
errors for the current bridge. That way, users can collaborate
on the same issue.
2019-06-08 17:05:35 +02:00
logmanoriginal
17f587fcbe [index] Don't set the timezone in index.php 2019-06-08 16:16:03 +02:00
logmanoriginal
f28cbecc02 [style] Fix placeholder should be hidden on focus
The placeholder is currently visible on key focus and only hidden
once a user starts typing. This can be confusing and doesn't look
good.

As it turns out, ::placeholder is an official selector:
https://developer.mozilla.org/en-US/docs/Web/CSS/::placeholder

For some reason, listing placeholder selectors with "," doesn't
work on some browsers (tested in FF 60 ESR). Making each of the
selectors explicit works, however.
2019-06-08 15:50:16 +02:00
LogMANOriginal
84450371b5 [README] Remove Deploy to Docker Cloud button
In December 2018 Docker Cloud has become part of Docker Hub:
https://blog.docker.com/2018/12/the-new-docker-hub/

Since then the "Deploy to Docker Cloud" button is broken (error 404)
with no alternative for Docker Hub, so the button should be removed.

Docker images are still available at
https://hub.docker.com/r/rssbridge/rss-bridge/
2019-06-08 15:19:56 +02:00
logmanoriginal
69dd33ac82 [.gitattributes] Use the same indentation style for the entire file 2019-06-08 15:07:08 +02:00
logmanoriginal
95388cdf44 [.gitattributes] Exclude demo bridges from release builds 2019-06-08 15:03:25 +02:00
logmanoriginal
b74dda7af9 [.gitattributes] Exclude Composer and Heroku files from release builds 2019-06-08 15:00:07 +02:00
logmanoriginal
ca1a5feba5 [.gitattributes] Annotate export-ignore sections 2019-06-08 14:58:18 +02:00
Squirrel
69a0498732 [README] Add deploy button to Heroku (#1150)
* Add deploy button to Heroku
* Add composer.json and composer.lock (required by Heroku)
2019-06-08 14:53:26 +02:00
logmanoriginal
3d231a417f bridges: Don't kill scripts with die()
Bridges should generally utilize the API functions instead of killing
the script. Find more information on the Wiki.

- returnServerError
https://github.com/RSS-Bridge/rss-bridge/wiki/The-returnServerError-function

- returnClientError
https://github.com/RSS-Bridge/rss-bridge/wiki/The-returnClientError-function

- returnError
https://github.com/RSS-Bridge/rss-bridge/wiki/The-returnError-function
2019-06-07 20:38:09 +02:00
logmanoriginal
35bd706391 [Configuration] Use common format to report errors to the user
Incorrect configuration values are currently handled individually
for each condition, resulting in a lot of repetitive operations.

This commit adds two new private functions to report errors to the
user and end execution of the script.
2019-06-07 20:27:20 +02:00
logmanoriginal
0e30468e0f [rssbridge] Use PATH_ROOT whenever possible 2019-06-07 19:51:06 +02:00
logmanoriginal
ccf375e917 config: Use global constant for config files
The configuration files are currently hard-coded in the configuration
classes and error messages. However, the implementation should not
rely on specific details like the file name. Instead, the files should
be part of the global definition.

This commit introduces two global constants for the configuration files

- FILE_CONFIG => 'config.ini.php'
- FILE_CONFIG_DEFAULT => 'config.default.ini.php'
2019-06-07 19:48:29 +02:00
logmanoriginal
946a99d334 config: Add [system] => 'timezone'
RSS-Bridge currently statically sets the timezone to UTC which can
result in incorrect timestamps if the server is hosted in another
region.

This commit adds a new configuration parameter to allow admins to
specify their own timezone for their servers. Invalid values will
result in an error message.

Example:

  [system]
  timezone = "UTC"

For compatibility reasons the default value is set to UTC.

This parameter accepts any of the supported timezones listed at
https://www.php.net/manual/en/timezones.php

Closes #956
References #1001
2019-06-07 19:22:51 +02:00
logmanoriginal
e2e0ced055 [Bridge] Improve performance for correctly written whitelist.txt
If the bridge name matches exactly, it is not necessary to perform
a strtolower compare of bridges. In some situations this can lead
to much faster response times (depending on the amount of bridges
in whitelist.txt).
2019-06-06 20:59:33 +02:00
logmanoriginal
d4e867f240 core: Move default bridges to whitelist.default.txt
Default bridges are currently statically defined in index.php, which
is not the right place if we want to keep responsibilities separated.

This commit introduces a new file whitelist.default.txt that holds
the default bridges and which is loaded automatically, if whitelist.txt
doesn't exist.

Due to this it is also no longer necessary to have write permission
for the root directory.

References #1001
2019-06-06 20:53:46 +02:00
Eugene Molotov
b0a780acda [VkBridge] Ignore illegal characters in input html for iconv (#1154) 2019-06-06 20:05:41 +02:00
Antoine Cadoret
1814116d67 [SteamBridge] Follow source changes (#1143)
* Follow source data fetching changes
* Improve media path building
* Improve price fetching and display
2019-06-06 19:59:30 +02:00
LogMANOriginal
d89326fe2d Remove old bridge request template 2019-06-06 19:57:04 +02:00
LogMANOriginal
62198ecfa2 Rename bridge request template
Use the same naming convention for all templates
2019-06-06 19:55:57 +02:00
LogMANOriginal
94e4ef8f27 Add template for generic feature requests 2019-06-06 19:54:34 +02:00
logmanoriginal
6c4098d655 Revert "all: Use ->remove() instead of ->outertext = ''"
This reverts commit 052844f5e1.

There is a bug in ->remove() that causes the parser to incorrectly
identify elements in the DOM tree that shouldn't exist anymore.

References #1151
2019-06-02 13:06:16 +02:00
logmanoriginal
468d8be72d [Exceptions] Fix GitHub query labels for bug reports
All bug reports now use the Bridge-Broken label by default
2019-06-01 22:35:56 +02:00
LogMANOriginal
ed539bacf9 Add issue template for generic bug reports
This commit adds a new template for generic bug reports based on the standard template provided by GitHub.
2019-06-01 22:35:33 +02:00
LogMANOriginal
82a9bb5b1c [.github] Update issue template for bridge requests
* Automatically label bridge requests
* Propose default title for new bridge requests
2019-06-01 22:22:05 +02:00
Eugene Molotov
15c374e317 [PikabuBridge] More options and fixes (#1149)
* Add gif support
* Use page title as feed title
* Implement community support
2019-06-01 21:35:18 +02:00
logmanoriginal
052844f5e1 all: Use ->remove() instead of ->outertext = ''
simplehtmldom 1.9 introduced new functions to recursively remove
nodes from the DOM. This allows removing elements without the need
to re-load the document by using $html->load($html->save()), which
is very inefficient.

Find more information about remove() at
https://simplehtmldom.sourceforge.io/docs/1.9/api/simple_html_dom_node/remove/
2019-06-01 21:29:57 +02:00
logmanoriginal
014b698f67 [html] Use find('*') over custom solution
find('*') wasn't supported in older versions of simplehtmldom but it
is supported now. Thus, all custom implementations can be replaced
by the correct solution.
2019-06-01 21:05:12 +02:00
logmanoriginal
5656792cee [simplehtmldom] Update to version 1.9
Find the release notes at
https://sourceforge.net/projects/simplehtmldom/files/simplehtmldom/1.9/
2019-06-01 20:02:07 +02:00
fulmeek
66c5b732cf [FeedItem] Avoid repeated UID hashing after loading from cache (#1148)
This fixes the following issue:

1. bridge sets unique ids for the items (ids get hashed)
2. items go to the cache
3. on next run items get loaded from cache
4. these items have different ids because they were hashed again
5. they show up twice in feed reader
2019-06-01 19:36:46 +02:00
Joseph
b889e867fd [SoundCloudBridge] Use account avatar as feed icon (#1146) 2019-06-01 15:04:42 +02:00
sysadminstory
b519d350bf [RadioMelodieBridge] Fix bridge after website update (#1145)
- The bridge has been adapted to the new website layout
- The content now shows the header picture below the date
2019-06-01 12:12:17 +02:00
Joseph
2a254855d8 [HaveIBeenPwnedBridge] Add new bridge (#1144) 2019-06-01 12:06:58 +02:00
Nemo
72bcc173eb [Docker] Switch Docker Image to official php base image (#1140)
* Switch Docker Image to official php base image

Switch from the unofficial Alpine+php image to the official php-apache image.
This has 2 advantages:

1. Official image is guaranteed to have regular updates, etc
2. The persistent Docker Alpine DNS Issue goes away;
https://github.com/gliderlabs/docker-alpine/issues/255

* [Docker] Ignore more files from Docker Image
2019-06-01 11:25:01 +02:00
Tobias Alexander Franke
4a60f05fd6 [BinanceBridge] Add new bridge (#1135) 2019-06-01 11:18:30 +02:00
somini
84d48d5614 [QPlayBridge]: New Bridge (#1118)
* [QPlayBridge]: New Bridge
2019-05-29 22:51:52 +02:00
Tobias Alexander Franke
7cf898b5af [SteamCommunityBridge] Add new bridge (#1136)
* [SteamCommunityBridge] Add new bridge
2019-05-29 22:50:04 +02:00
killruana
16bd2aec7a [MediapartBridge] Add new bridge (#1130)
* If no cookie session is defined, use the default rss stream
* Add a parameter for enabling/disabling the single page mode
2019-05-15 21:51:23 +02:00
Dreckiger-Dan
3d87ecbf8c [.gitignore] Add robots.txt to the ignore list (#1128) 2019-05-15 21:40:50 +02:00
Lyra
2cd310c025 Bump version to 2019-05-08 2019-05-08 22:36:22 +02:00
sysadminstory
b764204c3a [YoutubeBridge] Playlist bug fix (#1117)
This commit allow the bridge to parse an infinite number of items of a
Youtube playlist.

It should fix #647 !
2019-05-08 22:17:48 +02:00
Tobias Alexander Franke
a9e2574016 [ArtStationBridge] Added new bridge (#1122)
* [ArtStationBridge] Added new bridge
2019-05-08 22:14:53 +02:00
pofilo
e3f6e1c6db [DELETE] Deletion Google Plus bridge (#1124) 2019-05-08 22:11:50 +02:00
Lyra
8150a73922 [CourrierInternationalBridge] Use newer https-based URL 2019-05-08 22:09:49 +02:00
Lyra
a2f3866383 [RoadAndTrackBridge] Major rewrite, due to the depreciation of their API 2019-05-08 21:57:59 +02:00
Obsidienne
a3446ae77b [AO3Bridge] Add new bridge (#1123)
* [AO3Bridge] Add new bridge
2019-05-06 13:28:42 +02:00
Eugene Molotov
75359bc11b [core] Implemented MemcachedCache (#1000)
* [core] Implemented MemcachedCache
2019-05-03 11:56:07 +02:00
Roliga
fe103974f5 [BadDragonBridge] Add new bridge (#1082)
* [BadDragonBridge] Add new bridge
2019-05-02 22:02:13 +02:00
fulmeek
33c16f8be5 [BakaUpdatesMangaReleasesBridge] Sanitize hash for more solid UIDs (#1113)
This should minimize occasional hiccups on regular updates.
2019-04-30 21:01:48 +02:00
fulmeek
21d3bf3b60 caches: Refactor the API (#1060)
- For consistency, functions should always return null on non-existing data.

- WordPressPluginUpdateBridge appears to have used its own cache instance in the past. Obviously not used anymore.

- Since $key can be anything, the cache implementation must ensure to assign the related data reliably; most commonly by serializing and hashing the key in an appropriate way.

- Even though the default path for storage is perfectly fine, some people may want to use a different location. This is an example how a cache implementation is responsible for its requirements.
2019-04-29 20:12:43 +02:00
sysadminstory
3b8f3da09d [AutoJMBridge] Use title from website for Feed Title (#1093)
* [AutoJMBridge] Use title from website for Feed Title
2019-04-20 22:22:06 +02:00
sysadminstory
f9c4a84c25 [RadioMelodieBridge] Update to support new Website (#1101)
* [RadioMelodieBridge] Update to support new Website
2019-04-20 22:19:22 +02:00
Lorenzo Stanco
7b8dd93a8e [InstagramBridge] Fix image link 2019-04-20 22:15:30 +02:00
somini
8f5151b222 [SIMARBridge]: Add new bridge (#1055)
* [SIMARBridge]: Add new bridge
2019-04-16 09:58:22 +02:00
Lyra
98c2530984 [HDWallpapers] Adapt to some website changes (Fixes #1088). Add wallpapers to enclosures, and select "HD" as the default resolution 2019-04-07 22:02:11 +02:00
sysadminstory
90bf90d167 [BingSearch] Make the bridge compatible with PHP 5.6 (#1084)
* [BingSearch] Make the bridge compatible with PHP 5.6

The use of isset() with an expression is not possible in PHP 5.6. I
fixed it by replacing isset() with "null !== ".
2019-04-07 21:51:48 +02:00
Eugene Molotov
6feda2220e [VkBridge] Add option to hide reposts (#1089) 2019-04-07 21:50:58 +02:00
Lyra
92775abe11 Fix phpcs 2019-04-05 10:59:30 +02:00
Lyra
24cdeabed8 [GithubSearchBridge] Update the bridge to match Github's layout 2019-04-05 10:53:28 +02:00
Roliga
380fdf2e40 [ParameterValidator] Handle missing parameter type (#1057)
* [ParameterValidator] Handle missing parameter type
2019-04-04 22:55:46 +02:00
Tobias Alexander Franke
50c90eb5df [EconomistBridge] Add new bridge (#1067)
* [EconomistBridge] Added new bridge
2019-04-04 22:54:08 +02:00
DJCrashdummy
d9ee9e272e [FDroidBridge] fixed bridge (#1075)
because an additional widget (i guess the language selector) was added to the homepage.
2019-04-04 22:52:59 +02:00
Dreckiger-Dan
4ba0d8bebe Update .gitignore (#1078)
ignore .htaccess .htpasswd
2019-04-04 22:50:33 +02:00
somini
c9b0cd1315 ComboiosDePortugalBridge: HACK: Encode the URL (#1074)
This seems like a weird bug somewhere.

Either the HTML parser should return the valid page, or the CMS should
not convert the URL first, or the URL validation regex is buggy.
2019-04-04 22:48:25 +02:00
Lyra
2dc0c36e9b Merge branch 'master' of github.com:RSS-Bridge/rss-bridge 2019-04-04 22:46:49 +02:00
Lyra
0aa8858551 [RoadAndTrackBridge] Generate a signature key for every client instead of hardcoding it 2019-04-04 22:45:41 +02:00
Thibault Couraud
966d450d27 [FindACrew] Update bridge according new findacrew.net website (#1080)
* update bridge according new crewbay.com website
2019-04-04 22:44:44 +02:00
sysadminstory
291e8c2a23 [AutoJMBridge] Fix bridge after website change (#1081)
* [AutoJMBridge] Fix bridge after website change

The website was totally reworked, so the bridge had to be reworked too.
The bridge parameters changed, therefore old RSS feed will not work
anymore, but it was impossible to do it in another way.
2019-04-04 22:39:39 +02:00
DnAp
b6943de0ca [BingSearch] Add new bridge (#1046) 2019-03-23 16:40:19 +01:00
Aleś Bułojčyk
b9bbc9bdda [FacebookBridge] Fix decoding of cyrillic letters in group names (#842) 2019-03-23 16:39:09 +01:00
logmanoriginal
835e3b1163 [MozillaBugTrackerBridge] Fix typo 2019-03-23 16:30:15 +01:00
Antoine Turmel
3212156925 [MozillaBugTrackerBridge] New Bridge (#916)
This Bridge is a clone of KernelBugTrackerBridge but for Mozilla Bugzilla. There is some difference in the class used to get the right comments.
2019-03-23 16:27:07 +01:00
Dreckiger-Dan
281eaacaeb [HeiseBridge] Add new bridge (#744) 2019-03-23 16:22:44 +01:00
Xurxo Fresco
18d5ef192c [IvooxBridge] Add new bridge (#597) 2019-03-22 21:33:46 +01:00
logmanoriginal
6293c3d33d [FeedItem] Filter duplicate enclosures 2019-03-21 19:42:44 +01:00
logmanoriginal
835af1faf1 travis: Update build script to test more reasonable configurations
PHP nightly recently got updated to dev-8.x, which is not supported
by any of the test scripts. This makes the test pretty useless and
doesn't help in any way.

Instead, the build script should focus on current versions of PHP,
starting from 5.6 to 7.3 (current stable release).

PHP 7.3 is a reasonable version to use for finding breaking changes
in the test scripts (phpunit especially warns about changes). These
tests can fail, of course.
2019-03-20 19:31:34 +01:00
logmanoriginal
88aae6fd95 core: Apply changes to fix broken Travis builds
Travis-CI recently got updated, which causes existing builds to fail.
For example: https://travis-ci.org/RSS-Bridge/rss-bridge/builds/507568117

Indenting multi-line arguments of functions fixes it.
2019-03-20 19:23:22 +01:00
Nemo
684558e276 [StockFilingsBridge] Add new bridge (#1011) 2019-03-17 20:40:21 +01:00
logmanoriginal
d7094b7feb [Configuration] Bump version to dev.2019-03-17 2019-03-17 20:31:17 +01:00
71 changed files with 3352 additions and 1478 deletions

View File

@@ -1,7 +1,14 @@
.git
.gitattributes
.github/*
.travis.yml
cache/*
CONTRIBUTING.md
DEBUG
Dockerfile
whitelist.txt
phpcompatibility.xml
phpcs.xml
CONTRIBUTING.md
phpcs.xml
scalingo.json
tests/*
whitelist.txt

57
.gitattributes vendored
View File

@@ -10,27 +10,40 @@
*.dbproj merge=union
# Standard to msysgit
*.doc diff=astextplain
*.DOC diff=astextplain
*.docx diff=astextplain
*.DOCX diff=astextplain
*.dot diff=astextplain
*.DOT diff=astextplain
*.pdf diff=astextplain
*.PDF diff=astextplain
*.rtf diff=astextplain
*.RTF diff=astextplain
*.doc diff=astextplain
*.DOC diff=astextplain
*.docx diff=astextplain
*.DOCX diff=astextplain
*.dot diff=astextplain
*.DOT diff=astextplain
*.pdf diff=astextplain
*.PDF diff=astextplain
*.rtf diff=astextplain
*.RTF diff=astextplain
# Ignore files in git archive (i.e. GitHub release builds)
Dockerfile export-ignore
.travis.yml export-ignore
.github/ export-ignore
.gitattributes export-ignore
.gitignore export-ignore
.dockerignore export-ignore
scalingo.json export-ignore
phpunit.xml export-ignore
phpcs.xml export-ignore
phpcompatibility.xml export-ignore
tests/ export-ignore
cache/.gitkeep export-ignore
## Docker
Dockerfile export-ignore
.dockerignore export-ignore
## Travis
.travis.yml export-ignore
## GitHub
.github/ export-ignore
## Git
.gitattributes export-ignore
.gitignore export-ignore
## Scalingo
scalingo.json export-ignore
## RSS-Bridge
phpunit.xml export-ignore
phpcs.xml export-ignore
phpcompatibility.xml export-ignore
tests/ export-ignore
cache/.gitkeep export-ignore
bridges/DemoBridge.php export-ignore
bridges/FeedExpanderExampleBridge.php export-ignore
## Composer
composer.json export-ignore
composer.lock export-ignore
## Heroku
app.json export-ignore

View File

@@ -1,6 +1,9 @@
---
name: Bridge request template
name: Bridge request
about: Use this template for requesting a new bridge
title: Bridge request for ...
labels: Bridge-Request
assignees: ''
---

38
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@@ -0,0 +1,38 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: Bug-Report
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
**Smartphone (please complete the following information):**
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
**Additional context**
Add any other context about the problem here.

View File

@@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: Feature-Request
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.

7
.gitignore vendored
View File

@@ -236,3 +236,10 @@ config.ini.php
#Builder
.buildconfig
#Auth
.htaccess
.htpasswd
#Crawler
robots.txt

View File

@@ -4,41 +4,43 @@ language: php
install:
- composer global require dealerdirect/phpcodesniffer-composer-installer;
- composer global require phpcompatibility/php-compatibility;
# Use PHPUnit 6 for unit tests (stable), requires PHP 7
- if [[ $TRAVIS_PHP_VERSION == "7.0" ]]; then
composer global require phpunit/phpunit ^6;
fi
# Use latest PHPUnit on nightly to detect breaking changes
- if [[ $TRAVIS_PHP_VERSION == "nightly" ]]; then
composer global require phpunit/phpunit;
- if [[ "$PHPUNIT" ]]; then
composer global require phpunit/phpunit ^$PHPUNIT;
fi
script:
- phpenv rehash
# Run PHP_CodeSniffer on all versions
- ~/.config/composer/vendor/bin/phpcs . --standard=phpcs.xml --warning-severity=0 --extensions=php -p;
# Check PHP compatibility for the lowest supported version
- if [[ $TRAVIS_PHP_VERSION == "5.6" ]]; then
# Check PHP compatibility for the lowest and highest supported version
- if [[ $TRAVIS_PHP_VERSION == "5.6" || $TRAVIS_PHP_VERSION == "7.3" ]]; then
~/.config/composer/vendor/bin/phpcs . --standard=phpcompatibility.xml --extensions=php -p;
fi
# Run unit tests (stable)
- if [[ $TRAVIS_PHP_VERSION == "7.0" ]]; then
phpunit --configuration=phpunit.xml --include-path=lib/;
fi
# Run unit tests (latest/nightly)
# Check PHP compatibility for all versions, starting at the lowest supported version in order to detect breaking changes
- if [[ $TRAVIS_PHP_VERSION == "nightly" ]]; then
phpunit --configuration=phpunit.xml --include-path=lib/;
~/.config/composer/vendor/bin/phpcs . --standard=PHPCompatibility --extensions=php -p --runtime-set testVersion 5.6-;
# Run unit tests on highest major version
- if [[ ${TRAVIS_PHP_VERSION:0:1} == "7" ]]; then
~/.config/composer/vendor/bin/phpunit --configuration=phpunit.xml --include-path=lib/;
fi
php:
- 7.3
env:
- PHPUNIT=6
- PHPUNIT=7
- PHPUNIT=8
matrix:
fast_finish: true
include:
- php: 5.6
env: PHPUNIT=
- php: 7.0
- php: nightly
- php: 7.1
- php: 7.2
allow_failures:
- php: nightly
- php: 7.3
env: PHPUNIT=7
- php: 7.3
env: PHPUNIT=8

View File

@@ -1,5 +1,11 @@
FROM ulsmith/alpine-apache-php7
FROM php:7-apache
COPY ./ /app/public/
ENV APACHE_DOCUMENT_ROOT=/app
RUN chown -R apache:root /app/public
RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini" \
&& apt-get --yes update && apt-get --yes install libxml2-dev \
&& docker-php-ext-install -j$(nproc) simplexml \
&& sed -ri -e 's!/var/www/html!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/sites-available/*.conf \
&& sed -ri -e 's!/var/www/!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/apache2.conf /etc/apache2/conf-available/*.conf
COPY --chown=www-data:www-data ./ /app/

View File

@@ -85,7 +85,7 @@ Deploy
Thanks to the community, hosting your own instance of RSS-Bridge is as easy as clicking a button!
[![Deploy on Scalingo](https://cdn.scalingo.com/deploy/button.svg)](https://my.scalingo.com/deploy?source=https://github.com/sebsauvage/rss-bridge)
[![Deploy to Docker Cloud](https://files.cloud.docker.com/images/deploy-to-dockercloud.svg)](https://cloud.docker.com/stack/deploy/?repo=https://github.com/rss-bridge/rss-bridge)
[![Deploy to Heroku](https://www.herokucdn.com/deploy/button.svg)](https://heroku.com/deploy)
Getting involved
===
@@ -116,13 +116,14 @@ https://gist.github.com/LogMANOriginal/da00cd1e5f0ca31cef8e193509b17fd8
* [Ahiles3005](https://github.com/Ahiles3005)
* [Albirew](https://github.com/Albirew)
* [aledeg](https://github.com/aledeg)
* [alex73](https://github.com/alex73)
* [alexAubin](https://github.com/alexAubin)
* [AmauryCarrade](https://github.com/AmauryCarrade)
* [AntoineTurmel](https://github.com/AntoineTurmel)
* [ArthurHoaro](https://github.com/ArthurHoaro)
* [Astalaseven](https://github.com/Astalaseven)
* [Astyan-42](https://github.com/Astyan-42)
* [az5he6ch](https://github.com/az5he6ch)
* [azdkj532](https://github.com/azdkj532)
* [b1nj](https://github.com/b1nj)
* [benasse](https://github.com/benasse)
* [captn3m0](https://github.com/captn3m0)
@@ -134,8 +135,11 @@ https://gist.github.com/LogMANOriginal/da00cd1e5f0ca31cef8e193509b17fd8
* [da2x](https://github.com/da2x)
* [Daiyousei](https://github.com/Daiyousei)
* [disk0x](https://github.com/disk0x)
* [DJCrashdummy](https://github.com/DJCrashdummy)
* [Djuuu](https://github.com/Djuuu)
* [DnAp](https://github.com/DnAp)
* [Draeli](https://github.com/Draeli)
* [Dreckiger-Dan](https://github.com/Dreckiger-Dan)
* [em92](https://github.com/em92)
* [eMerzh](https://github.com/eMerzh)
* [EtienneM](https://github.com/EtienneM)
@@ -153,6 +157,7 @@ https://gist.github.com/LogMANOriginal/da00cd1e5f0ca31cef8e193509b17fd8
* [jdigilio](https://github.com/jdigilio)
* [JeremyRand](https://github.com/JeremyRand)
* [Jocker666z](https://github.com/Jocker666z)
* [killruana](https://github.com/killruana)
* [klimplant](https://github.com/klimplant)
* [kranack](https://github.com/kranack)
* [kraoc](https://github.com/kraoc)
@@ -170,7 +175,6 @@ https://gist.github.com/LogMANOriginal/da00cd1e5f0ca31cef8e193509b17fd8
* [mdemoss](https://github.com/mdemoss)
* [melangue](https://github.com/melangue)
* [metaMMA](https://github.com/metaMMA)
* [mickael-bertrand](https://github.com/mickael-bertrand)
* [mitsukarenai](https://github.com/mitsukarenai)
* [MonsieurPoutounours](https://github.com/MonsieurPoutounours)
* [mr-flibble](https://github.com/mr-flibble)
@@ -179,6 +183,7 @@ https://gist.github.com/LogMANOriginal/da00cd1e5f0ca31cef8e193509b17fd8
* [nel50n](https://github.com/nel50n)
* [niawag](https://github.com/niawag)
* [Nono-m0le](https://github.com/Nono-m0le)
* [ObsidianWitch](https://github.com/ObsidianWitch)
* [ORelio](https://github.com/ORelio)
* [PaulVayssiere](https://github.com/PaulVayssiere)
* [pellaeon](https://github.com/pellaeon)
@@ -186,6 +191,7 @@ https://gist.github.com/LogMANOriginal/da00cd1e5f0ca31cef8e193509b17fd8
* [pit-fgfjiudghdf](https://github.com/pit-fgfjiudghdf)
* [pitchoule](https://github.com/pitchoule)
* [pmaziere](https://github.com/pmaziere)
* [Pofilo](https://github.com/Pofilo)
* [prysme01](https://github.com/prysme01)
* [quentinus95](https://github.com/quentinus95)
* [regisenguehard](https://github.com/regisenguehard)
@@ -200,10 +206,13 @@ https://gist.github.com/LogMANOriginal/da00cd1e5f0ca31cef8e193509b17fd8
* [sysadminstory](https://github.com/sysadminstory)
* [tameroski](https://github.com/tameroski)
* [teromene](https://github.com/teromene)
* [thefranke](https://github.com/thefranke)
* [TheRadialActive](https://github.com/TheRadialActive)
* [triatic](https://github.com/triatic)
* [VerifiedJoseph](https://github.com/VerifiedJoseph)
* [WalterBarrett](https://github.com/WalterBarrett)
* [wtuuju](https://github.com/wtuuju)
* [xurxof](https://github.com/xurxof)
* [yardenac](https://github.com/yardenac)
* [ZeNairolf](https://github.com/ZeNairolf)

View File

@@ -86,9 +86,9 @@ class DisplayAction extends ActionAbstract {
// Initialize cache
$cache = Cache::create(Configuration::getConfig('cache', 'type'));
$cache->setPath(PATH_CACHE);
$cache->setScope('');
$cache->purgeCache(86400); // 24 hours
$cache->setParameters($cache_params);
$cache->setKey($cache_params);
$items = array();
$infos = array();

8
app.json Normal file
View File

@@ -0,0 +1,8 @@
{
"service": "Heroku",
"name": "RSS-Bridge",
"description": "RSS-Bridge is a PHP project capable of generating RSS and Atom feeds for websites which don't have one.",
"repository": "https://github.com/RSS-Bridge/rss-bridge",
"keywords": ["php", "rss-bridge", "rss"]
}

121
bridges/AO3Bridge.php Normal file
View File

@@ -0,0 +1,121 @@
<?php
class AO3Bridge extends BridgeAbstract {
const NAME = 'AO3';
const URI = 'https://archiveofourown.org/';
const CACHE_TIMEOUT = 1800;
const DESCRIPTION = 'Returns works or chapters from Archive of Our Own';
const MAINTAINER = 'Obsidienne';
const PARAMETERS = array(
'List' => array(
'url' => array(
'name' => 'url',
'required' => true,
// Example: F/F tag, complete works only
'exampleValue' => self::URI
. 'works?work_search[complete]=T&tag_id=F*s*F',
),
),
'Bookmarks' => array(
'user' => array(
'name' => 'user',
'required' => true,
// Example: Nyaaru's bookmarks
'exampleValue' => 'Nyaaru',
),
),
'Work' => array(
'id' => array(
'name' => 'id',
'required' => true,
// Example: latest chapters from A Better Past by LysSerris
'exampleValue' => '18181853',
),
)
);
// Feed for lists of works (e.g. recent works, search results, filtered tags,
// bookmarks, series, collections).
private function collectList($url) {
$html = getSimpleHTMLDOM($url)
or returnServerError('could not request AO3');
$html = defaultLinkTo($html, self::URI);
foreach($html->find('.index.group > li') as $element) {
$item = array();
$title = $element->find('div h4 a', 0);
if (!isset($title)) continue; // discard deleted works
$item['title'] = $title->plaintext;
$item['content'] = $element;
$item['uri'] = $title->href;
$strdate = $element->find('div p.datetime', 0)->plaintext;
$item['timestamp'] = strtotime($strdate);
$chapters = $element->find('dl dd.chapters', 0);
// bookmarked series and external works do not have a chapters count
$chapters = (isset($chapters) ? $chapters->plaintext : 0);
$item['uid'] = $item['uri'] . "/$strdate/$chapters";
$this->items[] = $item;
}
}
// Feed for recent chapters of a specific work.
private function collectWork($id) {
$url = self::URI . "/works/$id/navigate";
$html = getSimpleHTMLDOM($url)
or returnServerError('could not request AO3');
$html = defaultLinkTo($html, self::URI);
$this->title = $html->find('h2 a', 0)->plaintext;
foreach($html->find('ol.index.group > li') as $element) {
$item = array();
$item['title'] = $element->find('a', 0)->plaintext;
$item['content'] = $element;
$item['uri'] = $element->find('a', 0)->href;
$strdate = $element->find('span.datetime', 0)->plaintext;
$strdate = str_replace('(', '', $strdate);
$strdate = str_replace(')', '', $strdate);
$item['timestamp'] = strtotime($strdate);
$item['uid'] = $item['uri'] . "/$strdate";
$this->items[] = $item;
}
$this->items = array_reverse($this->items);
}
public function collectData() {
switch($this->queriedContext) {
case 'Bookmarks':
$user = $this->getInput('user');
$this->title = $user;
$url = self::URI
. '/users/' . $user
. '/bookmarks?bookmark_search[sort_column]=bookmarkable_date';
return $this->collectList($url);
case 'List': return $this->collectList(
$this->getInput('url')
);
case 'Work': return $this->collectWork(
$this->getInput('id')
);
}
}
public function getName() {
$name = parent::getName() . " $this->queriedContext";
if (isset($this->title)) $name .= " - $this->title";
return $name;
}
public function getIcon() {
return self::URI . '/favicon.ico';
}
}

View File

@@ -0,0 +1,93 @@
<?php
class ArtStationBridge extends BridgeAbstract {
const NAME = 'ArtStation';
const URI = 'https://www.artstation.com';
const DESCRIPTION = 'Fetches the latest ten artworks from a search query on ArtStation.';
const MAINTAINER = 'thefranke';
const CACHE_TIMEOUT = 3600; // 1h
const PARAMETERS = array(
'Search Query' => array(
'q' => array(
'name' => 'Search term',
'required' => true
)
)
);
public function getIcon() {
return 'https://www.artstation.com/assets/favicon-58653022bc38c1905ac7aa1b10bffa6b.ico';
}
public function getName() {
return self::NAME . ': ' . $this->getInput('q');
}
private function fetchSearch($searchQuery) {
$data = '{"query":"' . $searchQuery . '","page":1,"per_page":50,"sorting":"date",';
$data .= '"pro_first":"1","filters":[],"additional_fields":[]}';
$header = array(
'Content-Type: application/json',
'Accept: application/json'
);
$opts = array(
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => $data,
CURLOPT_RETURNTRANSFER => true
);
$jsonSearchURL = self::URI . '/api/v2/search/projects.json';
$jsonSearchStr = getContents($jsonSearchURL, $header, $opts)
or returnServerError('Could not fetch JSON for search query.');
return json_decode($jsonSearchStr);
}
private function fetchProject($hashID) {
$jsonProjectURL = self::URI . '/projects/' . $hashID . '.json';
$jsonProjectStr = getContents($jsonProjectURL)
or returnServerError('Could not fetch JSON for project.');
return json_decode($jsonProjectStr);
}
public function collectData() {
$searchTerm = $this->getInput('q');
$jsonQuery = $this->fetchSearch($searchTerm);
foreach($jsonQuery->data as $media) {
// get detailed info about media item
$jsonProject = $this->fetchProject($media->hash_id);
// create item
$item = array();
$item['title'] = $media->title;
$item['uri'] = $media->url;
$item['timestamp'] = strtotime($jsonProject->published_at);
$item['author'] = $media->user->full_name;
$item['categories'] = implode(',', $jsonProject->tags);
$item['content'] = '<a href="'
. $media->url
. '"><img style="max-width: 100%" src="'
. $jsonProject->cover_url
. '"></a><p>'
. $jsonProject->description
. '</p>';
$numAssets = count($jsonProject->assets);
if ($numAssets > 1)
$item['content'] .= '<p><a href="'
. $media->url
. '">Project contains '
. ($numAssets - 1)
. ' more item(s).</a></p>';
$this->items[] = $item;
if (count($this->items) >= 10)
break;
}
}
}

View File

@@ -91,7 +91,8 @@ class Arte7Bridge extends BridgeAbstract {
'Authorization: Bearer ' . self::API_TOKEN
);
$input = getContents($url, $header) or die('Could not request ARTE.');
$input = getContents($url, $header)
or returnServerError('Could not request ARTE.');
$input_json = json_decode($input, true);
foreach($input_json['videos'] as $element) {

View File

@@ -3,63 +3,195 @@
class AutoJMBridge extends BridgeAbstract {
const NAME = 'AutoJM';
const URI = 'http://www.autojm.fr/';
const URI = 'https://www.autojm.fr/';
const DESCRIPTION = 'Suivre les offres de véhicules proposés par AutoJM en fonction des critères de filtrages';
const MAINTAINER = 'sysadminstory';
const PARAMETERS = array(
'Afficher les offres de véhicules disponible en fonction des critères du site AutoJM' => array(
'url' => array(
'name' => 'URL de la recherche',
'name' => 'URL du modèle',
'type' => 'text',
'required' => true,
'title' => 'URL d\'une recherche avec filtre de véhicules sans le http://www.autojm.fr/',
'exampleValue' => 'gammes/index/398?order_by=finition_asc&energie[]=3&transmission[]=2&dispo=all'
'exampleValue' => 'achat-voitures-neuves-peugeot-nouvelle-308-5p'
),
'isDispo' => array(
'name' => 'Disponibilité',
'type' => 'list',
'values' => array(
'-' => '',
'En stock' => 1,
'Sur commande' => 0
),
'title' => 'Critère de disponibilité'
),
'energy' => array(
'name' => 'Carburant',
'type' => 'list',
'values' => array(
'-' => '',
'Diesel' => 1,
'Essence' => 3,
'Hybride' => 5
),
'title' => 'Carburant'
),
'transmission' => array(
'name' => 'Transmission',
'type' => 'list',
'values' => array(
'-' => '',
'Automatique' => 1,
'Manuelle' => 2
),
'title' => 'Transmission'
),
'priceMin' => array(
'name' => 'Prix minimum',
'type' => 'number',
'required' => false,
'title' => 'Prix minimum du véhicule',
'exampleValue' => '10000',
'defaultValue' => '0'
),
'priceMax' => array(
'name' => 'Prix maximum',
'type' => 'number',
'required' => false,
'title' => 'Prix maximum du véhicule',
'exampleValue' => '15000',
'defaultValue' => '150000'
)
)
);
const CACHE_TIMEOUT = 3600;
public function getIcon() {
return self::URI . 'assets/images/favicon.ico';
return self::URI . 'favicon.ico';
}
public function collectData() {
$html = getSimpleHTMLDOM(self::URI . $this->getInput('url'))
or returnServerError('Could not request AutoJM.');
$list = $html->find('div[class*=ligne_modele]');
foreach($list as $element) {
$image = $element->find('img[class=width-100]', 0)->src;
$serie = $element->find('div[class=serie]', 0)->find('span', 0)->plaintext;
$url = $element->find('div[class=serie]', 0)->find('a[class=btn_ligne color-black]', 0)->href;
if($element->find('div[class*=hasStock-info]', 0) != null) {
$dispo = 'Disponible';
} else {
$dispo = 'Sur commande';
}
$carburant = str_replace('dispo |', '', $element->find('div[class=carburant]', 0)->plaintext);
$transmission = $element->find('div[class*=bv]', 0)->plaintext;
$places = $element->find('div[class*=places]', 0)->plaintext;
$portes = $element->find('div[class*=nb_portes]', 0)->plaintext;
$carosserie = $element->find('div[class*=coloris]', 0)->plaintext;
$remise = $element->find('div[class*=remise]', 0)->plaintext;
$prix = $element->find('div[class*=prixjm]', 0)->plaintext;
$item = array();
$item['uri'] = $url;
$item['title'] = $serie;
$item['content'] = '<p><img style="vertical-align:middle ; padding: 10px" src="' . $image . '" />' . $serie . '</p>';
$item['content'] .= '<ul><li>Disponibilité : ' . $dispo . '</li>';
$item['content'] .= '<li>Carburant : ' . $carburant . '</li>';
$item['content'] .= '<li>Transmission : ' . $transmission . '</li>';
$item['content'] .= '<li>Nombre de places : ' . $places . '</li>';
$item['content'] .= '<li>Nombre de portes : ' . $portes . '</li>';
$item['content'] .= '<li>Série : ' . $serie . '</li>';
$item['content'] .= '<li>Carosserie : ' . $carosserie . '</li>';
$item['content'] .= '<li>Remise : ' . $remise . '</li>';
$item['content'] .= '<li>Prix : ' . $prix . '</li></ul>';
$this->items[] = $item;
public function getName() {
switch($this->queriedContext) {
case 'Afficher les offres de véhicules disponible en fonction des critères du site AutoJM':
$html = getSimpleHTMLDOMCached(self::URI . $this->getInput('url'), 86400);
$name = html_entity_decode($html->find('title', 0)->plaintext);
return $name;
break;
default:
return parent::getName();
}
}
public function collectData() {
$model_url = self::URI . $this->getInput('url');
// Get the session cookies and the form token
$this->getInitialParameters($model_url);
// Build the form
$post_data = array(
'form[isDispo]' => $this->getInput('isDispo'),
'form[energy]' => $this->getInput('energy'),
'form[transmission]' => $this->getInput('transmission'),
'form[priceMin]' => $this->getInput('priceMin'),
'form[priceMin]' => $this->getInput('priceMin'),
'form[_token]' => $this->token
);
// Set the Form request content type
$header = array(
'Content-Type: application/x-www-form-urlencoded; charset=UTF-8',
);
// Set the curl options (POST query and content, and session cookies
$curl_opts = array(
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => http_build_query($post_data),
CURLOPT_COOKIE => $this->cookies
);
// Get the JSON content of the form
$json = getContents($model_url, $header, $curl_opts)
or returnServerError('Could not request AutoJM.');
// Extract the HTML content from the JSON result
$data = json_decode($json);
$html = str_get_html($data->content);
// Go through every finisha of the model
$list = $html->find('h2');
foreach ($list as $finish) {
$finish_name = $finish->plaintext;
$motorizations = $finish->next_sibling()->find('li');
foreach ($motorizations as $element) {
$image = $element->find('div[class=block-product-image]', 0)->{'data-ga-banner'};
$serie = $element->find('span[class=model]', 0)->plaintext;
$url = self::URI . substr($element->find('a', 0)->href, 1);
if ($element->find('span[class*=block-product-nbModel]', 0) != null) {
$availability = 'En Stock';
} else {
$availability = 'Sur commande';
}
$discount_html = $element->find('span[class*=tag--promo]', 0);
if ($discount_html != null) {
$discount = $discount_html->plaintext;
} else {
$discount = 'inconnue';
}
$price = $element->find('span[class=price red h1]', 0)->plaintext;
$item = array();
$item['title'] = $finish_name . ' ' . $serie;
$item['content'] = '<p><img style="vertical-align:middle ; padding: 10px" src="' . $image . '" />'
. $finish_name . ' ' . $serie . '</p>';
$item['content'] .= '<ul><li>Disponibilité : ' . $availability . '</li>';
$item['content'] .= '<li>Série : ' . $serie . '</li>';
$item['content'] .= '<li>Remise : ' . $discount . '</li>';
$item['content'] .= '<li>Prix : ' . $price . '</li></ul>';
// Add a fictionnal anchor to the RSS element URL, based on the item content ;
// As the URL could be identical even if the price change, some RSS reader will not show those offers as new items
$item['uri'] = $url . '#' . md5($item['content']);
$this->items[] = $item;
}
}
}
/**
* Gets the session cookie and the form token
*
* @param string $pageURL The URL from which to get the values
*/
private function getInitialParameters($pageURL) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $pageURL);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$data = curl_exec($ch);
// Separate the response header and the content
$headerSize = curl_getinfo($ch, CURLINFO_HEADER_SIZE);
$header = substr($data, 0, $headerSize);
$content = substr($data, $headerSize);
curl_close($ch);
// Extract the cookies from the headers
$cookies = '';
$http_response_header = explode("\r\n", $header);
foreach ($http_response_header as $hdr) {
if (strpos($hdr, 'Set-Cookie') !== false) {
$cLine = explode(':', $hdr)[1];
$cLine = explode(';', $cLine)[0];
$cookies .= ';' . $cLine;
}
}
$this->cookies = trim(substr($cookies, 1));
// Get the token from the content
$html = str_get_html($content);
$token = $html->find('input[type=hidden][id=form__token]', 0);
$this->token = $token->value;
}
}

435
bridges/BadDragonBridge.php Normal file
View File

@@ -0,0 +1,435 @@
<?php
class BadDragonBridge extends BridgeAbstract {
const NAME = 'Bad Dragon Bridge';
const URI = 'https://bad-dragon.com/';
const CACHE_TIMEOUT = 300; // 5min
const DESCRIPTION = 'Returns sales or new clearance items';
const MAINTAINER = 'Roliga';
const PARAMETERS = array(
'Sales' => array(
),
'Clearance' => array(
'ready_made' => array(
'name' => 'Ready Made',
'type' => 'checkbox'
),
'flop' => array(
'name' => 'Flops',
'type' => 'checkbox'
),
'skus' => array(
'name' => 'Products',
'exampleValue' => 'chanceflared, crackers',
'title' => 'Comma separated list of product SKUs'
),
'onesize' => array(
'name' => 'One-Size',
'type' => 'checkbox'
),
'mini' => array(
'name' => 'Mini',
'type' => 'checkbox'
),
'small' => array(
'name' => 'Small',
'type' => 'checkbox'
),
'medium' => array(
'name' => 'Medium',
'type' => 'checkbox'
),
'large' => array(
'name' => 'Large',
'type' => 'checkbox'
),
'extralarge' => array(
'name' => 'Extra Large',
'type' => 'checkbox'
),
'category' => array(
'name' => 'Category',
'type' => 'list',
'values' => array(
'All' => 'all',
'Accessories' => 'accessories',
'Merchandise' => 'merchandise',
'Dildos' => 'insertable',
'Masturbators' => 'penetrable',
'Packers' => 'packer',
'Lil\' Squirts' => 'shooter',
'Lil\' Vibes' => 'vibrator',
'Wearables' => 'wearable'
),
'defaultValue' => 'all',
),
'soft' => array(
'name' => 'Soft Firmness',
'type' => 'checkbox'
),
'med_firm' => array(
'name' => 'Medium Firmness',
'type' => 'checkbox'
),
'firm' => array(
'name' => 'Firm',
'type' => 'checkbox'
),
'split' => array(
'name' => 'Split Firmness',
'type' => 'checkbox'
),
'maxprice' => array(
'name' => 'Max Price',
'type' => 'number',
'required' => true,
'defaultValue' => 300
),
'minprice' => array(
'name' => 'Min Price',
'type' => 'number',
'defaultValue' => 0
),
'cumtube' => array(
'name' => 'Cumtube',
'type' => 'checkbox'
),
'suctionCup' => array(
'name' => 'Suction Cup',
'type' => 'checkbox'
),
'noAccessories' => array(
'name' => 'No Accessories',
'type' => 'checkbox'
)
)
);
/*
* This sets index $strFrom (or $strTo if set) in $outArr to 'on' if
* $inArr[$param] contains $strFrom.
* It is used for translating BD's shop filter URLs into something we can use.
*
* For the query '?type[]=ready_made&type[]=flop' we would have an array like:
* Array (
* [type] => Array (
* [0] => ready_made
* [1] => flop
* )
* )
* which could be translated into:
* Array (
* [ready_made] => on
* [flop] => on
* )
* */
private function setParam($inArr, &$outArr, $param, $strFrom, $strTo = null) {
if(isset($inArr[$param]) && in_array($strFrom, $inArr[$param])) {
$outArr[($strTo ?: $strFrom)] = 'on';
}
}
public function detectParameters($url) {
$params = array();
// Sale
$regex = '/^(https?:\/\/)?bad-dragon\.com\/sales/';
if(preg_match($regex, $url, $matches) > 0) {
return $params;
}
// Clearance
$regex = '/^(https?:\/\/)?bad-dragon\.com\/shop\/clearance/';
if(preg_match($regex, $url, $matches) > 0) {
parse_str(parse_url($url, PHP_URL_QUERY), $urlParams);
$this->setParam($urlParams, $params, 'type', 'ready_made');
$this->setParam($urlParams, $params, 'type', 'flop');
if(isset($urlParams['skus'])) {
$skus = array();
foreach($urlParams['skus'] as $sku) {
is_string($sku) && $skus[] = $sku;
is_array($sku) && $skus[] = $sku[0];
}
$params['skus'] = implode(',', $skus);
}
$this->setParam($urlParams, $params, 'sizes', 'onesize');
$this->setParam($urlParams, $params, 'sizes', 'mini');
$this->setParam($urlParams, $params, 'sizes', 'small');
$this->setParam($urlParams, $params, 'sizes', 'medium');
$this->setParam($urlParams, $params, 'sizes', 'large');
$this->setParam($urlParams, $params, 'sizes', 'extralarge');
if(isset($urlParams['category'])) {
$params['category'] = strtolower($urlParams['category']);
} else{
$params['category'] = 'all';
}
$this->setParam($urlParams, $params, 'firmnessValues', 'soft');
$this->setParam($urlParams, $params, 'firmnessValues', 'medium', 'med_firm');
$this->setParam($urlParams, $params, 'firmnessValues', 'firm');
$this->setParam($urlParams, $params, 'firmnessValues', 'split');
if(isset($urlParams['price'])) {
isset($urlParams['price']['max'])
&& $params['maxprice'] = $urlParams['price']['max'];
isset($urlParams['price']['min'])
&& $params['minprice'] = $urlParams['price']['min'];
}
isset($urlParams['cumtube'])
&& $urlParams['cumtube'] === '1'
&& $params['cumtube'] = 'on';
isset($urlParams['suctionCup'])
&& $urlParams['suctionCup'] === '1'
&& $params['suctionCup'] = 'on';
isset($urlParams['noAccessories'])
&& $urlParams['noAccessories'] === '1'
&& $params['noAccessories'] = 'on';
return $params;
}
return null;
}
public function getName() {
switch($this->queriedContext) {
case 'Sales':
return 'Bad Dragon Sales';
case 'Clearance':
return 'Bad Dragon Clearance Search';
default:
return parent::getName();
}
}
public function getURI() {
switch($this->queriedContext) {
case 'Sales':
return self::URI . 'sales';
case 'Clearance':
return $this->inputToURL();
default:
return parent::getURI();
}
}
public function collectData() {
switch($this->queriedContext) {
case 'Sales':
$sales = json_decode(getContents(self::URI . 'api/sales'))
or returnServerError('Failed to query BD API');
foreach($sales as $sale) {
$item = array();
$item['title'] = $sale->title;
$item['timestamp'] = strtotime($sale->startDate);
$item['uri'] = $this->getURI() . '/' . $sale->slug;
$contentHTML = '<p><img src="' . $sale->image->url . '"></p>';
if(isset($sale->endDate)) {
$contentHTML .= '<p><b>This promotion ends on '
. gmdate('M j, Y \a\t g:i A T', strtotime($sale->endDate))
. '</b></p>';
} else {
$contentHTML .= '<p><b>This promotion never ends</b></p>';
}
$ul = false;
$content = json_decode($sale->content);
foreach($content->blocks as $block) {
switch($block->type) {
case 'header-one':
$contentHTML .= '<h1>' . $block->text . '</h1>';
break;
case 'header-two':
$contentHTML .= '<h2>' . $block->text . '</h2>';
break;
case 'header-three':
$contentHTML .= '<h3>' . $block->text . '</h3>';
break;
case 'unordered-list-item':
if(!$ul) {
$contentHTML .= '<ul>';
$ul = true;
}
$contentHTML .= '<li>' . $block->text . '</li>';
break;
default:
if($ul) {
$contentHTML .= '</ul>';
$ul = false;
}
$contentHTML .= '<p>' . $block->text . '</p>';
break;
}
}
$item['content'] = $contentHTML;
$this->items[] = $item;
}
break;
case 'Clearance':
$toyData = json_decode(getContents($this->inputToURL(true)))
or returnServerError('Failed to query BD API');
$productList = json_decode(getContents(self::URI
. 'api/inventory-toy/product-list'))
or returnServerError('Failed to query BD API');
foreach($toyData->toys as $toy) {
$item = array();
$item['uri'] = $this->getURI()
. '#'
. $toy->id;
$item['timestamp'] = strtotime($toy->created);
foreach($productList as $product) {
if($product->sku == $toy->sku) {
$item['title'] = $product->name;
break;
}
}
// images
$content = '<p>';
foreach($toy->images as $image) {
$content .= '<a href="'
. $image->fullFilename
. '"><img src="'
. $image->thumbFilename
. '" /></a>';
}
// price
$content .= '</p><p><b>Price:</b> $'
. $toy->price
// size
. '<br /><b>Size:</b> '
. $toy->size
// color
. '<br /><b>Color:</b> '
. $toy->color
// features
. '<br /><b>Features:</b> '
. ($toy->suction_cup ? 'Suction cup' : '')
. ($toy->suction_cup && $toy->cumtube ? ', ' : '')
. ($toy->cumtube ? 'Cumtube' : '')
. ($toy->suction_cup || $toy->cumtube ? '' : 'None');
// firmness
$firmnessTexts = array(
'2' => 'Extra soft',
'3' => 'Soft',
'5' => 'Medium',
'8' => 'Firm'
);
$firmnesses = explode('/', $toy->firmness);
if(count($firmnesses) === 2) {
$content .= '<br /><b>Firmness:</b> '
. $firmnessTexts[$firmnesses[0]]
. ', '
. $firmnessTexts[$firmnesses[1]];
} else{
$content .= '<br /><b>Firmness:</b> '
. $firmnessTexts[$firmnesses[0]];
}
// flop
if($toy->type === 'flop') {
$content .= '<br /><b>Flop reason:</b> '
. $toy->flop_reason;
}
$content .= '</p>';
$item['content'] = $content;
$enclosures = array();
foreach($toy->images as $image) {
$enclosures[] = $image->fullFilename;
}
$item['enclosures'] = $enclosures;
$categories = array();
$categories[] = $toy->sku;
$categories[] = $toy->type;
$categories[] = $toy->size;
if($toy->cumtube) {
$categories[] = 'cumtube';
}
if($toy->suction_cup) {
$categories[] = 'suction_cup';
}
$item['categories'] = $categories;
$this->items[] = $item;
}
break;
}
}
private function inputToURL($api = false) {
$url = self::URI;
$url .= ($api ? 'api/inventory-toys?' : 'shop/clearance?');
// Default parameters
$url .= 'limit=60';
$url .= '&page=1';
$url .= '&sort[field]=created';
$url .= '&sort[direction]=desc';
// Product types
$url .= ($this->getInput('ready_made') ? '&type[]=ready_made' : '');
$url .= ($this->getInput('flop') ? '&type[]=flop' : '');
// Product names
foreach(array_filter(explode(',', $this->getInput('skus'))) as $sku) {
$url .= '&skus[]=' . urlencode(trim($sku));
}
// Size
$url .= ($this->getInput('onesize') ? '&sizes[]=onesize' : '');
$url .= ($this->getInput('mini') ? '&sizes[]=mini' : '');
$url .= ($this->getInput('small') ? '&sizes[]=small' : '');
$url .= ($this->getInput('medium') ? '&sizes[]=medium' : '');
$url .= ($this->getInput('large') ? '&sizes[]=large' : '');
$url .= ($this->getInput('extralarge') ? '&sizes[]=extralarge' : '');
// Category
$url .= ($this->getInput('category') ? '&category='
. urlencode($this->getInput('category')) : '');
// Firmness
if($api) {
$url .= ($this->getInput('soft') ? '&firmnessValues[]=3' : '');
$url .= ($this->getInput('med_firm') ? '&firmnessValues[]=5' : '');
$url .= ($this->getInput('firm') ? '&firmnessValues[]=8' : '');
if($this->getInput('split')) {
$url .= '&firmnessValues[]=3/5';
$url .= '&firmnessValues[]=3/8';
$url .= '&firmnessValues[]=8/3';
$url .= '&firmnessValues[]=5/8';
$url .= '&firmnessValues[]=8/5';
}
} else{
$url .= ($this->getInput('soft') ? '&firmnessValues[]=soft' : '');
$url .= ($this->getInput('med_firm') ? '&firmnessValues[]=medium' : '');
$url .= ($this->getInput('firm') ? '&firmnessValues[]=firm' : '');
$url .= ($this->getInput('split') ? '&firmnessValues[]=split' : '');
}
// Price
$url .= ($this->getInput('maxprice') ? '&price[max]='
. $this->getInput('maxprice') : '&price[max]=300');
$url .= ($this->getInput('minprice') ? '&price[min]='
. $this->getInput('minprice') : '&price[min]=0');
// Features
$url .= ($this->getInput('cumtube') ? '&cumtube=1' : '');
$url .= ($this->getInput('suctionCup') ? '&suctionCup=1' : '');
$url .= ($this->getInput('noAccessories') ? '&noAccessories=1' : '');
return $url;
}
}

View File

@@ -68,7 +68,7 @@ class BakaUpdatesMangaReleasesBridge extends BridgeAbstract {
$item['title'] = implode(' ', $title);
$item['uri'] = $this->getURI();
$item['uid'] = hash('sha1', $item['title']);
$item['uid'] = $this->getSanitizedHash($item['title']);
$this->items[] = $item;
}
@@ -89,8 +89,12 @@ class BakaUpdatesMangaReleasesBridge extends BridgeAbstract {
return parent::getName();
}
private function getSanitizedHash($string) {
return hash('sha1', preg_replace('/[^a-zA-Z0-9\-\.]/', '', ucwords(strtolower($string))));
}
private function filterText($text) {
return rtrim($text, '*');
return rtrim($text, '* ');
}
private function filterHTML($text) {

103
bridges/BinanceBridge.php Normal file
View File

@@ -0,0 +1,103 @@
<?php
class BinanceBridge extends BridgeAbstract {
const NAME = 'Binance';
const URI = 'https://www.binance.com';
const DESCRIPTION = 'Subscribe to the Binance blog or the Binance Zendesk announcements.';
const MAINTAINER = 'thefranke';
const CACHE_TIMEOUT = 3600; // 1h
const PARAMETERS = array( array(
'category' => array(
'name' => 'category',
'type' => 'list',
'exampleValue' => 'Blog',
'title' => 'Select a category',
'values' => array(
'Blog' => 'Blog',
'Announcements' => 'Announcements'
)
)
));
public function getIcon() {
return 'https://bin.bnbstatic.com/static/images/common/favicon.ico';
}
public function getName() {
return self::NAME . ' ' . $this->getInput('category');
}
public function getURI() {
if ($this->getInput('category') == 'Blog')
return self::URI . '/en/blog';
else
return 'https://binance.zendesk.com/hc/en-us/categories/115000056351-Announcements';
}
protected function collectBlogData() {
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not fetch Binance blog data.');
foreach($html->find('div[direction="row"]') as $element) {
$date = $element->find('div[direction="column"]', 0);
$day = $date->find('div', 0)->innertext;
$month = $date->find('div', 1)->innertext;
$extractedDate = $day . ' ' . $month;
$abstract = $element->find('div[direction="column"]', 1);
$a = $abstract->find('a', 0);
$uri = self::URI . $a->href;
$title = $a->innertext;
$full = getSimpleHTMLDOMCached($uri);
$content = $full->find('div.desc', 1);
$item = array();
$item['title'] = $title;
$item['uri'] = $uri;
$item['timestamp'] = strtotime($extractedDate);
$item['author'] = 'Binance';
$item['content'] = $content;
$this->items[] = $item;
if (count($this->items) >= 10)
break;
}
}
protected function collectAnnouncementData() {
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not fetch Zendesk announcement data.');
foreach($html->find('a.article-list-link') as $a) {
$title = $a->innertext;
$uri = 'https://binance.zendesk.com' . $a->href;
$full = getSimpleHTMLDOMCached($uri);
$content = $full->find('div.article-body', 0);
$date = $full->find('time', 0)->getAttribute('datetime');
$item = array();
$item['title'] = $title;
$item['uri'] = $uri;
$item['timestamp'] = strtotime($date);
$item['author'] = 'Binance';
$item['content'] = $content;
$this->items[] = $item;
if (count($this->items) >= 10)
break;
}
}
public function collectData() {
if ($this->getInput('category') == 'Blog')
$this->collectBlogData();
else
$this->collectAnnouncementData();
}
}

View File

@@ -0,0 +1,119 @@
<?php
class BingSearchBridge extends BridgeAbstract
{
const NAME = 'Bing search';
const URI = 'https://www.bing.com/';
const DESCRIPTION = 'Return images from bing search discover';
const MAINTAINER = 'DnAp';
const PARAMETERS = array(
'Image Discover' => array(
'category' => array(
'name' => 'Categories',
'type' => 'list',
'values' => self::IMAGE_DISCOVER_CATEGORIES
),
'image_size' => array(
'name' => 'Image size',
'type' => 'list',
'values' => array(
'Small' => 'turl',
'Full size' => 'imgurl'
)
)
)
);
const IMAGE_DISCOVER_CATEGORIES = array(
'Abstract' => 'abstract',
'Animals' => 'animals',
'Anime' => 'anime',
'Architecture' => 'architecture',
'Arts and Crafts' => 'arts-and-crafts',
'Beauty' => 'beauty',
'Cars and Motorcycles' => 'cars-and-motorcycles',
'Cats' => 'cats',
'Celebrities' => 'celebrities',
'Comics' => 'comics',
'DIY' => 'diy',
'Dogs' => 'dogs',
'Fitness' => 'fitness',
'Food and Drink' => 'food-and-drink',
'Funny' => 'funny',
'Gadgets' => 'gadgets',
'Gardening' => 'gardening',
'Geeky' => 'geeky',
'Hairstyles' => 'hairstyles',
'Home Decor' => 'home-decor',
'Marine Life' => 'marine-life',
'Men\'s Fashion' => 'men%27s-fashion',
'Nature' => 'nature',
'Outdoors' => 'outdoors',
'Parenting' => 'parenting',
'Phone Wallpapers' => 'phone-wallpapers',
'Photography' => 'photography',
'Quotes' => 'quotes',
'Recipes' => 'recipes',
'Snow' => 'snow',
'Tattoos' => 'tattoos',
'Travel' => 'travel',
'Video Games' => 'video-games',
'Weddings' => 'weddings',
'Women\'s Fashion' => 'women%27s-fashion',
);
public function getIcon()
{
return 'https://www.bing.com/sa/simg/bing_p_rr_teal_min.ico';
}
public function collectData()
{
$this->items = $this->imageDiscover($this->getInput('category'));
}
public function getName()
{
if ($this->getInput('category')) {
if (self::IMAGE_DISCOVER_CATEGORIES[$this->getInput('categories')] !== null) {
$category = self::IMAGE_DISCOVER_CATEGORIES[$this->getInput('categories')];
} else {
$category = 'Unknown';
}
return 'Best ' . $category . ' - Bing Image Discover';
}
return parent::getName();
}
private function imageDiscover($category)
{
$html = getSimpleHTMLDOM(self::URI . '/discover/' . $category)
or returnServerError('Could not request ' . self::NAME);
$sizeKey = $this->getInput('image_size');
$items = [];
foreach ($html->find('a.iusc') as $element) {
$data = json_decode(htmlspecialchars_decode($element->getAttribute('m')), true);
$item = array();
$item['title'] = basename(rtrim($data['imgurl'], '/'));
$item['uri'] = $data['imgurl'];
$item['content'] = '<a href="' . $data['imgurl'] . '">
<img src="' . $data[$sizeKey] . '" alt="' . $item['title'] . '"></a>
<p>Source: <a href="' . $this->curUrl($data['surl']) . '"> </a></p>';
$item['enclosures'] = $data['imgurl'];
$items[] = $item;
}
return $items;
}
private function curUrl($url)
{
if (strlen($url) <= 80) {
return $url;
}
return substr($url, 0, 80) . '...';
}
}

142
bridges/BrutBridge.php Normal file
View File

@@ -0,0 +1,142 @@
<?php
class BrutBridge extends BridgeAbstract {
const NAME = 'Brut Bridge';
const URI = 'https://www.brut.media';
const DESCRIPTION = 'Returns 5 newest videos by category and edition';
const MAINTAINER = 'VerifiedJoseph';
const PARAMETERS = array(array(
'category' => array(
'name' => 'Category',
'type' => 'list',
'values' => array(
'News' => 'news',
'International' => 'international',
'Economy' => 'economy',
'Science and Technology' => 'science-and-technology',
'Entertainment' => 'entertainment',
'Sports' => 'sport',
'Nature' => 'nature',
),
'defaultValue' => 'news',
),
'edition' => array(
'name' => ' Edition',
'type' => 'list',
'values' => array(
'United States' => 'us',
'United Kingdom' => 'uk',
'France' => 'fr',
'India' => 'in',
'Mexico' => 'mx',
),
'defaultValue' => 'us',
)
)
);
const CACHE_TIMEOUT = 1800; // 30 mins
private $videoId = '';
private $videoType = '';
private $videoImage = '';
public function collectData() {
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request: ' . $this->getURI());
$results = $html->find('div.results', 0);
foreach($results->find('li.col-6.col-sm-4.col-md-3.col-lg-2.px-2.pb-4') as $index => $li) {
$item = array();
$videoPath = self::URI . $li->children(0)->href;
$videoPageHtml = getSimpleHTMLDOMCached($videoPath, 3600)
or returnServerError('Could not request: ' . $videoPath);
$this->videoImage = $videoPageHtml->find('meta[name="twitter:image"]', 0)->content;
$this->processTwitterImage();
$description = $videoPageHtml->find('div.description', 0);
$item['uri'] = $videoPath;
$item['title'] = $description->find('h1', 0)->plaintext;
if ($description->find('div.date', 0)->children(0)) {
$description->find('div.date', 0)->children(0)->outertext = '';
}
$item['content'] = $this->processContent(
$description
);
$item['timestamp'] = $this->processDate($description);
$item['enclosures'][] = $this->videoImage;
$this->items[] = $item;
if (count($this->items) >= 5) {
break;
}
}
}
public function getURI() {
if (!is_null($this->getInput('edition')) && !is_null($this->getInput('category'))) {
return self::URI . '/' . $this->getInput('edition') . '/' . $this->getInput('category');
}
return parent::getURI();
}
private function processDate($description) {
if ($this->getInput('edition') === 'uk') {
$date = DateTime::createFromFormat('d/m/Y H:i', $description->find('div.date', 0)->innertext);
return strtotime($date->format('Y-m-d H:i:s'));
}
return strtotime($description->find('div.date', 0)->innertext);
}
private function processContent($description) {
$content = '<video controls poster="' . $this->videoImage . '" preload="none">
<source src="https://content.brut.media/video/' . $this->videoId . '-' . $this->videoType . '-web.mp4"
type="video/mp4">
</video>';
$content .= '<p>' . $description->find('h2.mb-1', 0)->innertext . '</p>';
if ($description->find('div.text.pb-3', 0)->children(1)->class != 'date') {
$content .= '<p>' . $description->find('div.text.pb-3', 0)->children(1)->innertext . '</p>';
}
return $content;
}
private function processTwitterImage() {
/**
* Extract video ID + type from twitter image
*
* Example (wrapped):
* https://img.brut.media/thumbnail/
* the-life-of-rita-moreno-2cce75b5-d448-44d2-a97c-ca50d6470dd4-square.jpg
* ?ts=1559337892
*/
$fpath = parse_url($this->videoImage, PHP_URL_PATH);
$fname = basename($fpath);
$fname = substr($fname, 0, strrpos($fname, '.'));
$parts = explode('-', $fname);
if (end($parts) === 'auto') {
$key = array_search('auto', $parts);
unset($parts[$key]);
}
$this->videoId = implode('-', array_splice($parts, -6, 5));
$this->videoType = end($parts);
}
}

View File

@@ -14,7 +14,7 @@ class ComboiosDePortugalBridge extends BridgeAbstract {
$item = array();
$item['title'] = $element->innertext;
$item['uri'] = self::BASE_URI . $element->href;
$item['uri'] = self::BASE_URI . implode('/', array_map('urlencode', explode('/', $element->href)));
$this->items[] = $item;
}

View File

@@ -3,7 +3,7 @@ class CourrierInternationalBridge extends BridgeAbstract {
const MAINTAINER = 'teromene';
const NAME = 'Courrier International Bridge';
const URI = 'http://CourrierInternational.com/';
const URI = 'https://www.courrierinternational.com/';
const CACHE_TIMEOUT = 300; // 5 min
const DESCRIPTION = 'Courrier International bridge';

View File

@@ -0,0 +1,63 @@
<?php
class EconomistBridge extends BridgeAbstract {
const NAME = 'The Economist: Latest Updates';
const URI = 'https://www.economist.com';
const DESCRIPTION = 'Fetches the latest updates from the Economist.';
const MAINTAINER = 'thefranke';
const CACHE_TIMEOUT = 3600; // 1h
public function getIcon() {
return 'https://www.economist.com/sites/default/files/econfinal_favicon.ico';
}
public function collectData() {
$html = getSimpleHTMLDOM(self::URI . '/latest/')
or returnServerError('Could not fetch latest updates form The Economist.');
foreach($html->find('article') as $element) {
$a = $element->find('a', 0);
$href = self::URI . $a->href;
$full = getSimpleHTMLDOMCached($href);
$article = $full->find('article', 0);
$header = $article->find('h1', 0);
$author = $article->find('span[itemprop="author"]', 0);
$time = $article->find('time[itemprop="dateCreated"]', 0);
$content = $article->find('div[itemprop="description"]', 0);
// Remove newsletter subscription box
$newsletter = $content->find('div[class="newsletter-form__message"]', 0);
if ($newsletter)
$newsletter->outertext = '';
$newsletterForm = $content->find('form', 0);
if ($newsletterForm)
$newsletterForm->outertext = '';
// Remove next and previous article URLs at the bottom
$nextprev = $content->find('div[class="blog-post__next-previous-wrapper"]', 0);
if ($nextprev)
$nextprev->outertext = '';
$section = [ $article->find('h3[itemprop="articleSection"]', 0)->plaintext ];
$item = array();
$item['title'] = $header->find('span', 0)->innertext . ': '
. $header->find('span', 1)->innertext;
$item['uri'] = $href;
$item['timestamp'] = strtotime($time->datetime);
$item['author'] = $author->innertext;
$item['categories'] = $section;
$item['content'] = '<img style="max-width: 100%" src="'
. $a->find('img', 0)->src . '">' . $content->innertext;
$this->items[] = $item;
if (count($this->items) >= 10)
break;
}
}
}

View File

@@ -121,8 +121,8 @@ class ElloBridge extends BridgeAbstract {
private function getAPIKey() {
$cache = Cache::create(Configuration::getConfig('cache', 'type'));
$cache->setPath(PATH_CACHE);
$cache->setParameters(['key']);
$cache->setScope(get_called_class());
$cache->setKey(['key']);
$key = $cache->loadData();
if($key == null) {

View File

@@ -72,15 +72,15 @@ class FB2Bridge extends BridgeAbstract {
$pageInfo = $this->getPageInfos($page, $cookies);
if($pageInfo['userId'] === null) {
echo <<<EOD
returnClientError(<<<EOD
Unable to get the page id. You should consider getting the ID by hand, then importing it into FB2Bridge
EOD;
die();
EOD
);
} elseif($pageInfo['userId'] == -1) {
echo <<<EOD
returnClientError(<<<EOD
This page is not accessible without being logged in.
EOD;
die();
EOD
);
}
}
@@ -95,7 +95,7 @@ EOD;
foreach($html->find('article') as $content) {
$item = array();
//echo $content; die();
preg_match('/publish_time\\\":([0-9]+),/', $content->getAttribute('data-store', 0), $match);
if(isset($match[1]))
$timestamp = $match[1];

View File

@@ -28,14 +28,14 @@ class FDroidBridge extends BridgeAbstract {
or returnServerError('Could not request F-Droid.');
// targetting the corresponding widget based on user selection
// "updated" is the 4th widget on the page, "added" is the 5th
// "updated" is the 5th widget on the page, "added" is the 6th
switch($this->getInput('u')) {
case 'updated':
$html_widget = $html->find('div.sidebar-widget', 4);
$html_widget = $html->find('div.sidebar-widget', 5);
break;
default:
$html_widget = $html->find('div.sidebar-widget', 5);
$html_widget = $html->find('div.sidebar-widget', 6);
break;
}

View File

@@ -219,8 +219,7 @@ class FacebookBridge extends BridgeAbstract {
$ogtitle = $html->find('meta[property="og:title"]', 0)
or returnServerError('Unable to find group title!');
return htmlspecialchars_decode($ogtitle->content, ENT_QUOTES);
return html_entity_decode($ogtitle->content, ENT_QUOTES);
}
private function extractGroupURI($post) {

View File

@@ -62,10 +62,10 @@ class FindACrewBridge extends BridgeAbstract {
foreach ($annonces as $annonce) {
$item = array();
$img = parent::getURI() . $annonce->find('.css_LstPic img', 0)->getAttribute('src');
$item['title'] = $annonce->find('.css_LstCtrls span', 0)->plaintext;
$item['uri'] = parent::getURI() . $annonce->find('.css_PnlCtrls a', 0)->href;
$content = $annonce->find('.css_LstDtl div', 2)->innertext;
$img = parent::getURI() . $annonce->find('.lst-pic img', 0)->getAttribute('src');
$item['title'] = $annonce->find('.lst-tags span', 0)->plaintext;
$item['uri'] = parent::getURI() . $annonce->find('.lst-ctrls a', 0)->href;
$content = $annonce->find('.lst-dtl', 0)->innertext;
$item['content'] = "<img src='$img' /><br>$content";
$item['enclosures'] = array($img);
$item['categories'] = array($annonce->find('.css_AccLocCur', 0)->plaintext);

View File

@@ -8,8 +8,8 @@ class GOGBridge extends BridgeAbstract {
public function collectData() {
$values = getContents('https://www.gog.com/games/ajax/filtered?limit=25&sort=new') or
die('Unable to get the news pages from GOG !');
$values = getContents('https://www.gog.com/games/ajax/filtered?limit=25&sort=new')
or returnServerError('Unable to get the news pages from GOG !');
$decodedValues = json_decode($values);
$limit = 0;
@@ -38,8 +38,8 @@ class GOGBridge extends BridgeAbstract {
private function buildGameContentPage($game) {
$gameDescriptionText = getContents('https://api.gog.com/products/' . $game->id . '?expand=description') or
die('Unable to get game description from GOG !');
$gameDescriptionText = getContents('https://api.gog.com/products/' . $game->id . '?expand=description')
or returnServerError('Unable to get game description from GOG !');
$gameDescriptionValue = json_decode($gameDescriptionText);

View File

@@ -24,7 +24,7 @@ class GithubSearchBridge extends BridgeAbstract {
$html = getSimpleHTMLDOM($url)
or returnServerError('Error while downloading the website content');
foreach($html->find('div.repo-list-item') as $element) {
foreach($html->find('li.repo-list-item') as $element) {
$item = array();
$uri = $element->find('h3 a', 0)->href;

View File

@@ -1,208 +0,0 @@
<?php
class GooglePlusPostBridge extends BridgeAbstract{
private $title;
private $url;
const MAINTAINER = 'Grummfy, logmanoriginal';
const NAME = 'Google Plus Post Bridge';
const URI = 'https://plus.google.com';
const CACHE_TIMEOUT = 600; //10min
const DESCRIPTION = 'Returns user public post (without API).';
const PARAMETERS = array( array(
'username' => array(
'name' => 'username or Id',
'required' => true
),
'include_media' => array(
'name' => 'Include media',
'type' => 'checkbox',
'title' => 'Enable to include media in the feed content'
)
));
public function getIcon() {
return 'https://ssl.gstatic.com/images/branding/product/ico/google_plus_alldp.ico';
}
public function collectData(){
$username = $this->getInput('username');
// Usernames start with a + if it's not an ID
if(!is_numeric($username) && substr($username, 0, 1) !== '+') {
$username = '+' . $username;
}
$html = getSimpleHTMLDOM(static::URI . '/' . urlencode($username) . '/posts')
or returnServerError('No results for this query.');
$html = defaultLinkTo($html, static::URI);
$this->title = $html->find('meta[property=og:title]', 0)->getAttribute('content');
$this->url = $html->find('meta[property=og:url]', 0)->getAttribute('content');
foreach($html->find('div[jsname=WsjYwc]') as $post) {
$item = array();
$item['author'] = $post->find('div div div div a', 0)->innertext;
$item['uri'] = $post->find('div div div a', 1)->href;
$timestamp = $post->find('a.qXj2He span', 0);
if($timestamp) {
$item['timestamp'] = strtotime('+' . preg_replace(
'/[^0-9A-Za-z]/',
'',
$timestamp->getAttribute('aria-label')));
}
$message = $post->find('div[jsname=EjRJtf]', 0);
// Empty messages are not supported right now
if(!$message) {
continue;
}
$item['content'] = '<div style="float: left; padding: 0 10px 10px 0;"><a href="'
. $this->url
. '"><img align="top" alt="'
. $item['author']
. '" src="'
. $post->find('div img', 0)->src
. '" /></a></div><div>'
. trim(strip_tags($message, '<a><p><div><img>'))
. '</div>';
// Make title at least 50 characters long, but don't add '...' if it is shorter!
if(strlen($message->plaintext) > 50) {
$end = strpos($message->plaintext, ' ', 50) ?: strlen($message->plaintext);
} else {
$end = strlen($message->plaintext);
}
if(strlen(substr($message->plaintext, 0, $end)) === strlen($message->plaintext)) {
$item['title'] = $message->plaintext;
} else {
$item['title'] = substr($message->plaintext, 0, $end) . '...';
}
$media = $post->find('[jsname="MTOxpb"]', 0);
if($media) {
$item['enclosures'] = array();
foreach($media->find('img') as $img) {
$item['enclosures'][] = $this->fixImage($img)->src;
}
if($this->getInput('include_media') === true && count($item['enclosures'] > 0)) {
$item['content'] .= '<div style="clear: both;"><a href="'
. $item['enclosures'][0]
. '"><img src="'
. $item['enclosures'][0]
. '" /></a></div>';
}
}
// Add custom parameters (only useful for JSON or Plaintext)
$item['fullname'] = $item['author'];
$item['avatar'] = $post->find('div img', 0)->src;
$item['id'] = $post->find('div div div', 0)->getAttribute('id');
$item['content_simple'] = $message->plaintext;
$this->items[] = $item;
}
}
public function getName(){
return $this->title ?: 'Google Plus Post Bridge';
}
public function getURI(){
return $this->url ?: parent::getURI();
}
private function fixImage($img) {
// There are certain images like .gif which link to a static picture and
// get replaced dynamically via JS in the browser. If we want the "real"
// image we need to account for that.
$urlparts = parse_url($img->src);
if(array_key_exists('host', $urlparts)) {
// For some reason some URIs don't contain the scheme, assume https
if(!array_key_exists('scheme', $urlparts)) {
$urlparts['scheme'] = 'https';
}
$pathelements = explode('/', $urlparts['path']);
switch($urlparts['host']) {
case 'lh3.googleusercontent.com':
if(pathinfo(end($pathelements), PATHINFO_EXTENSION)) {
// The second to last element of the path specifies the
// image format. The URL is still valid if we remove it.
unset($pathelements[count($pathelements) - 2]);
} elseif(strrpos(end($pathelements), '=') !== false) {
// Some images go throug a proxy. For those images they
// add size information after an equal sign.
// Example: '=w530-h298-n'. Again this can safely be
// removed to get the original image.
$pathelements[count($pathelements) - 1] = substr(
end($pathelements),
0,
strrpos(end($pathelements), '=')
);
}
break;
}
$urlparts['path'] = implode('/', $pathelements);
}
$img->src = $this->build_url($urlparts);
return $img;
}
/**
* From: https://gist.github.com/Ellrion/f51ba0d40ae1d62eeae44fd1adf7b704
* slightly adjusted to work with PHP < 7.0
* @param array $parts
* @return string
*/
private function build_url(array $parts)
{
$scheme = isset($parts['scheme']) ? ($parts['scheme'] . '://') : '';
$host = isset($parts['host']) ? $parts['host'] : '';
$port = isset($parts['port']) ? (':' . $parts['port']) : '';
$user = isset($parts['user']) ? $parts['user'] : '';
$pass = isset($parts['pass']) ? (':' . $parts['pass']) : '';
$pass = ($user || $pass) ? ($pass . '@') : '';
$path = isset($parts['path']) ? $parts['path'] : '';
$query = isset($parts['query']) ? ('?' . $parts['query']) : '';
$fragment = isset($parts['fragment']) ? ('#' . $parts['fragment']) : '';
return implode('', [$scheme, $user, $pass, $host, $port, $path, $query, $fragment]);
}
}

View File

@@ -16,13 +16,13 @@ class HDWallpapersBridge extends BridgeAbstract {
),
'r' => array(
'name' => 'resolution',
'defaultValue' => '1920x1200',
'exampleValue' => '1920x1200, 1680x1050,…'
'defaultValue' => 'HD',
'exampleValue' => 'HD, 1920x1200, 1680x1050,…'
)
));
public function collectData(){
$category = $this->category;
$category = $this->getInput('c');
if(strrpos($category, 'wallpapers') !== strlen($category) - strlen('wallpapers')) {
$category .= '-desktop-wallpapers';
}
@@ -45,13 +45,12 @@ class HDWallpapersBridge extends BridgeAbstract {
$thumbnail = $element->find('img', 0);
$item = array();
// http://www.hdwallpapers.in/download/yosemite_reflections-1680x1050.jpg
$item['uri'] = self::URI
. '/download'
. str_replace('wallpapers.html', $this->getInput('r') . '.jpg', $element->href);
$item['timestamp'] = time();
$item['title'] = $element->find('p', 0)->text();
$item['title'] = $element->find('em1', 0)->text();
$item['content'] = $item['title']
. '<br><a href="'
. $item['uri']
@@ -60,6 +59,7 @@ class HDWallpapersBridge extends BridgeAbstract {
. $thumbnail->src
. '" /></a>';
$item['enclosures'] = array($item['uri']);
$this->items[] = $item;
$num++;

View File

@@ -0,0 +1,102 @@
<?php
class HaveIBeenPwnedBridge extends BridgeAbstract {
const NAME = 'Have I Been Pwned (HIBP) Bridge';
const URI = 'https://haveibeenpwned.com';
const DESCRIPTION = 'Returns list of Pwned websites';
const MAINTAINER = 'VerifiedJoseph';
const PARAMETERS = array(array(
'order' => array(
'name' => 'Order by',
'type' => 'list',
'values' => array(
'Breach date' => 'breachDate',
'Date added to HIBP' => 'dateAdded',
),
'defaultValue' => 'dateAdded',
)
));
const CACHE_TIMEOUT = 3600;
private $breachDateRegex = '/Breach date: ([0-9]{1,2} [A-Z-a-z]+ [0-9]{4})/';
private $dateAddedRegex = '/Date added to HIBP: ([0-9]{1,2} [A-Z-a-z]+ [0-9]{4})/';
private $accountsRegex = '/Compromised accounts: ([0-9,]+)/';
private $breaches = array();
public function collectData() {
$html = getSimpleHTMLDOM(self::URI . '/PwnedWebsites')
or returnServerError('Could not request: ' . self::URI . '/PwnedWebsites');
$breaches = array();
foreach($html->find('div.row') as $breach) {
$item = array();
if ($breach->class != 'row') {
continue;
}
preg_match($this->breachDateRegex, $breach->find('p', 1)->plaintext, $breachDate)
or returnServerError('Could not extract details');
preg_match($this->dateAddedRegex, $breach->find('p', 1)->plaintext, $dateAdded)
or returnServerError('Could not extract details');
preg_match($this->accountsRegex, $breach->find('p', 1)->plaintext, $accounts)
or returnServerError('Could not extract details');
$permalink = $breach->find('p', 1)->find('a', 0)->href;
// Remove permalink
$breach->find('p', 1)->find('a', 0)->outertext = '';
$item['title'] = $breach->find('h3', 0)->plaintext . ' - ' . $accounts[1] . ' breached accounts';
$item['dateAdded'] = strtotime($dateAdded[1]);
$item['breachDate'] = strtotime($breachDate[1]);
$item['uri'] = self::URI . '/PwnedWebsites' . $permalink;
$item['content'] = '<p>' . $breach->find('p', 0)->innertext . '<p>';
$item['content'] .= '<p>' . $breach->find('p', 1)->innertext . '<p>';
$this->breaches[] = $item;
}
$this->orderBreaches();
$this->createItems();
}
/**
* Order Breaches by date added or date breached
*/
private function orderBreaches() {
$sortBy = $this->getInput('order');
$sort = array();
foreach ($this->breaches as $key => $item) {
$sort[$key] = $item[$sortBy];
}
array_multisort($sort, SORT_DESC, $this->breaches);
}
/**
* Create items from breaches array
*/
private function createItems() {
foreach ($this->breaches as $breach) {
$item = array();
$item['title'] = $breach['title'];
$item['timestamp'] = $breach[$this->getInput('order')];
$item['uri'] = $breach['uri'];
$item['content'] = $breach['content'];
$this->items[] = $item;
}
}
}

75
bridges/HeiseBridge.php Normal file
View File

@@ -0,0 +1,75 @@
<?php
class HeiseBridge extends FeedExpander {
const MAINTAINER = 'Dreckiger-Dan';
const NAME = 'Heise Online Bridge';
const URI = 'https://heise.de/';
const CACHE_TIMEOUT = 1800; // 30min
const DESCRIPTION = 'Returns the full articles instead of only the intro';
const PARAMETERS = array(array(
'category' => array(
'name' => 'Category',
'type' => 'list',
'values' => array(
'Alle News'
=> 'https://www.heise.de/newsticker/heise-atom.xml',
'Top-News'
=> 'https://www.heise.de/newsticker/heise-top-atom.xml',
'Internet-Störungen'
=> 'https://www.heise.de/netze/netzwerk-tools/imonitor-internet-stoerungen/feed/aktuelle-meldungen/',
'Alle News von heise Developer'
=> 'https://www.heise.de/developer/rss/news-atom.xml'
)
),
'limit' => array(
'name' => 'Limit',
'type' => 'number',
'required' => false,
'title' => 'Specify number of full articles to return',
'defaultValue' => 5
)
));
const LIMIT = 5;
public function collectData() {
$this->collectExpandableDatas(
$this->getInput('category'),
$this->getInput('limit') ?: static::LIMIT
);
}
protected function parseItem($feedItem) {
$item = parent::parseItem($feedItem);
$uri = $item['uri'];
do {
$article = getSimpleHTMLDOMCached($uri)
or returnServerError('Could not open article: ' . $uri);
$article = defaultLinkTo($article, $uri);
$item = $this->addArticleToItem($item, $article);
if($next = $article->find('.pagination a[rel="next"]', 0))
$uri = $next->href;
} while ($next);
return $item;
}
private function addArticleToItem($item, $article) {
if($author = $article->find('[itemprop="author"]', 0))
$item['author'] = $author->plaintext;
$content = $article->find('div[class*="article-content"]', 0);
foreach($content->find('p, h3, ul, table, pre, img') as $element) {
$item['content'] .= $element;
}
foreach($content->find('img') as $img) {
$item['enclosures'][] = $img->src;
}
return $item;
}
}

View File

@@ -89,7 +89,7 @@ class InstagramBridge extends BridgeAbstract {
if (isset($media->edge_media_to_caption->edges[0]->node->text)) {
$textContent = $media->edge_media_to_caption->edges[0]->node->text;
} else {
$textContent = basename($media->display_url);
$textContent = '(no text)';
}
$item['title'] = ($media->is_video ? '▶ ' : '') . trim($textContent);
@@ -103,10 +103,11 @@ class InstagramBridge extends BridgeAbstract {
$item['content'] = $data[0];
$item['enclosures'] = $data[1];
} else {
$mediaURI = self::URI . 'p/' . $media->shortcode . '/media?size=l';
$item['content'] = '<a href="' . htmlentities($item['uri']) . '" target="_blank">';
$item['content'] .= '<img src="' . htmlentities($media->display_url) . '" alt="' . $item['title'] . '" />';
$item['content'] .= '<img src="' . htmlentities($mediaURI) . '" alt="' . $item['title'] . '" />';
$item['content'] .= '</a><br><br>' . nl2br(htmlentities($textContent));
$item['enclosures'] = array($media->display_url);
$item['enclosures'] = array($mediaURI);
}
$item['timestamp'] = $media->taken_at_timestamp;

128
bridges/IvooxBridge.php Normal file
View File

@@ -0,0 +1,128 @@
<?php
/**
* IvooxRssBridge
* Returns the latest search result
* TODO: support podcast episodes list
*/
class IvooxBridge extends BridgeAbstract {
const NAME = 'Ivoox Bridge';
const URI = 'https://www.ivoox.com/';
const CACHE_TIMEOUT = 10800; // 3h
const DESCRIPTION = 'Returns the 10 newest episodes by keyword search';
const MAINTAINER = 'xurxof'; // based on YoutubeBridge by mitsukarenai
const PARAMETERS = array(
'Search result' => array(
's' => array(
'name' => 'keyword',
'exampleValue' => 'test'
)
)
);
private function ivBridgeAddItem(
$episode_link,
$podcast_name,
$episode_title,
$author_name,
$episode_description,
$publication_date,
$episode_duration) {
$item = array();
$item['title'] = htmlspecialchars_decode($podcast_name . ': ' . $episode_title);
$item['author'] = $author_name;
$item['timestamp'] = $publication_date;
$item['uri'] = $episode_link;
$item['content'] = '<a href="' . $episode_link . '">' . $podcast_name . ': ' . $episode_title
. '</a><br />Duration: ' . $episode_duration
. '<br />Description:<br />' . $episode_description;
$this->items[] = $item;
}
private function ivBridgeParseHtmlListing($html) {
$limit = 4;
$count = 0;
foreach($html->find('div.flip-container') as $flipper) {
$linkcount = 0;
if(!empty($flipper->find( 'div.modulo-type-banner' ))) {
// ad
continue;
}
if($count < $limit) {
foreach($flipper->find('div.header-modulo') as $element) {
foreach($element->find('a') as $link) {
if ($linkcount == 0) {
$episode_link = $link->href;
$episode_title = $link->title;
} elseif ($linkcount == 1) {
$author_link = $link->href;
$author_name = $link->title;
} elseif ($linkcount == 2) {
$podcast_link = $link->href;
$podcast_name = $link->title;
}
$linkcount++;
}
}
$episode_description = $flipper->find('button.btn-link', 0)->getAttribute('data-content');
$episode_duration = $flipper->find('p.time', 0)->innertext;
$publication_date = $flipper->find('li.date', 0)->getAttribute('title');
// alternative date_parse_from_format
// or DateTime::createFromFormat('G:i - d \d\e M \d\e Y', $publication);
// TODO: month name translations, due function doesn't support locale
$a = strptime($publication_date, '%H:%M - %d de %b. de %Y'); // obsolete function, uses c libraries
$publication_date = mktime(0, 0, 0, $a['tm_mon'] + 1, $a['tm_mday'], $a['tm_year'] + 1900);
$this->ivBridgeAddItem(
$episode_link,
$podcast_name,
$episode_title,
$author_name,
$episode_description,
$publication_date,
$episode_duration
);
$count++;
}
}
}
public function collectData() {
// store locale, change to spanish
$originalLocales = explode(';', setlocale(LC_ALL, 0));
setlocale(LC_ALL, 'es_ES.utf8');
$xml = '';
$html = '';
$url_feed = '';
if($this->getInput('s')) { /* Search modes */
$this->request = str_replace(' ', '-', $this->getInput('s'));
$url_feed = self::URI . urlencode($this->request) . '_sb_f_1.html?o=uploaddate';
} else {
returnClientError('Not valid mode at IvooxBridge');
}
$dom = getSimpleHTMLDOM($url_feed)
or returnServerError('Could not request ' . $url_feed);
$this->ivBridgeParseHtmlListing($dom);
// restore locale
foreach($originalLocales as $localeSetting) {
if(strpos($localeSetting, '=') !== false) {
list($category, $locale) = explode('=', $localeSetting);
} else {
$category = LC_ALL;
$locale = $localeSetting;
}
setlocale($category, $locale);
}
}
}

View File

@@ -0,0 +1,60 @@
<?php
class MediapartBridge extends FeedExpander {
const MAINTAINER = 'killruana';
const NAME = 'Mediapart Bridge';
const URI = 'https://www.mediapart.fr/';
const PARAMETERS = array(
array(
'single_page_mode' => array(
'name' => 'Single page article',
'type' => 'checkbox',
'title' => 'Display long articles on a single page',
'defaultValue' => 'checked'
),
'mpsessid' => array(
'name' => 'MPSESSID',
'type' => 'text',
'title' => 'Value of the session cookie MPSESSID'
)
)
);
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = 'Returns the newest articles.';
public function collectData() {
$url = self::URI . 'articles/feed';
$this->collectExpandableDatas($url);
}
protected function parseItem($newsItem) {
$item = parent::parseItem($newsItem);
// Enable single page mode?
if ($this->getInput('single_page_mode') === true) {
$item['uri'] .= '?onglet=full';
}
// If a session cookie is defined, get the full article
$mpsessid = $this->getInput('mpsessid');
if (!empty($mpsessid)) {
// Set the session cookie
$opt = array();
$opt[CURLOPT_COOKIE] = 'MPSESSID=' . $mpsessid;
// Get the page
$articlePage = getSimpleHTMLDOM(
$newsItem->link . '?onglet=full',
array(),
$opt);
// Extract the article content
$content = $articlePage->find('div.content-article', 0)->innertext;
$content = sanitize($content);
$content = defaultLinkTo($content, static::URI);
$item['content'] .= $content;
}
return $item;
}
}

View File

@@ -0,0 +1,153 @@
<?php
class MozillaBugTrackerBridge extends BridgeAbstract {
const NAME = 'Mozilla Bug Tracker';
const URI = 'https://bugzilla.mozilla.org';
const DESCRIPTION = 'Returns feeds for bug comments';
const MAINTAINER = 'AntoineTurmel';
const PARAMETERS = array(
'Bug comments' => array(
'id' => array(
'name' => 'Bug tracking ID',
'type' => 'number',
'required' => true,
'title' => 'Insert bug tracking ID',
'exampleValue' => 121241
),
'limit' => array(
'name' => 'Number of comments to return',
'type' => 'number',
'required' => false,
'title' => 'Specify number of comments to return',
'defaultValue' => -1
),
'sorting' => array(
'name' => 'Sorting',
'type' => 'list',
'required' => false,
'title' => 'Defines the sorting order of the comments returned',
'defaultValue' => 'of',
'values' => array(
'Oldest first' => 'of',
'Latest first' => 'lf'
)
)
)
);
private $bugid = '';
private $bugdesc = '';
public function getIcon() {
return self::URI . '/extensions/BMO/web/images/favicon.ico';
}
public function collectData(){
$limit = $this->getInput('limit');
$sorting = $this->getInput('sorting');
// We use the print preview page for simplicity
$html = getSimpleHTMLDOMCached($this->getURI() . '&format=multiple',
86400,
null,
null,
true,
true,
DEFAULT_TARGET_CHARSET,
false, // Do NOT remove line breaks
DEFAULT_BR_TEXT,
DEFAULT_SPAN_TEXT);
if($html === false)
returnServerError('Failed to load page!');
// Store header information into private members
$this->bugid = $html->find('#bugzilla-body', 0)->find('a', 0)->innertext;
$this->bugdesc = $html->find('table.bugfields', 0)->find('tr', 0)->find('td', 0)->innertext;
// Get and limit comments
$comments = $html->find('.bz_comment_table div.bz_comment');
if($limit > 0 && count($comments) > $limit) {
$comments = array_slice($comments, count($comments) - $limit, $limit);
}
// Order comments
switch($sorting) {
case 'lf': $comments = array_reverse($comments, true);
case 'of':
default: // Nothing to do, keep original order
}
foreach($comments as $comment) {
$comment = $this->inlineStyles($comment);
$item = array();
$item['uri'] = $this->getURI() . '#' . $comment->id;
$item['author'] = $comment->find('span.bz_comment_user', 0)->innertext;
$item['title'] = $comment->find('span.bz_comment_number', 0)->find('a', 0)->innertext;
$item['timestamp'] = strtotime($comment->find('span.bz_comment_time', 0)->innertext);
$item['content'] = $comment->find('pre.bz_comment_text', 0)->innertext;
// Fix line breaks (they use LF)
$item['content'] = str_replace("\n", '<br>', $item['content']);
// Fix relative URIs
$item['content'] = $this->replaceRelativeURI($item['content']);
$this->items[] = $item;
}
}
public function getURI(){
switch($this->queriedContext) {
case 'Bug comments':
return parent::getURI()
. '/show_bug.cgi?id='
. $this->getInput('id');
break;
default: return parent::getURI();
}
}
public function getName(){
switch($this->queriedContext) {
case 'Bug comments':
return 'Bug '
. $this->bugid
. ' tracker for '
. $this->bugdesc
. ' - '
. parent::getName();
break;
default: return parent::getName();
}
}
/**
* Replaces all relative URIs with absolute ones
*
* @param string $content The source string
* @return string Returns the source string with all relative URIs replaced
* by absolute ones.
*/
private function replaceRelativeURI($content){
return preg_replace('/href="(?!http)/', 'href="' . self::URI . '/', $content);
}
/**
* Adds styles as attributes to tags with known classes
*
* @param object $html A simplehtmldom object
* @return object Returns the original object with styles added as
* attributes.
*/
private function inlineStyles($html){
foreach($html->find('.bz_obsolete') as $element) {
$element->style = 'text-decoration:line-through;';
}
return $html;
}
}

View File

@@ -6,6 +6,16 @@ class PikabuBridge extends BridgeAbstract {
const DESCRIPTION = 'Выводит посты по тегу';
const MAINTAINER = 'em92';
const PARAMETERS_FILTER = array(
'name' => 'Фильтр',
'type' => 'list',
'values' => array(
'Горячее' => 'hot',
'Свежее' => 'new',
),
'defaultValue' => 'hot'
);
const PARAMETERS = array(
'По тегу' => array(
'tag' => array(
@@ -13,21 +23,29 @@ class PikabuBridge extends BridgeAbstract {
'exampleValue' => 'it',
'required' => true
),
'filter' => array(
'name' => 'Фильтр',
'type' => 'list',
'values' => array(
'Горячее' => 'hot',
'Свежее' => 'new',
),
'defaultValue' => 'hot'
)
'filter' => self::PARAMETERS_FILTER
),
'По сообществу' => array(
'community' => array(
'name' => 'Сообщество',
'exampleValue' => 'linux',
'required' => true
),
'filter' => self::PARAMETERS_FILTER
)
);
protected $title = null;
public function getURI() {
if ($this->getInput('tag')) {
return self::URI . '/tag/' . rawurlencode($this->getInput('tag')) . '/' . rawurlencode($this->getInput('filter'));
} else if ($this->getInput('community')) {
$uri = self::URI . '/community/' . rawurlencode($this->getInput('community'));
if ($this->getInput('filter') != 'hot') {
$uri .= '/' . rawurlencode($this->getInput('filter'));
}
return $uri;
} else {
return parent::getURI();
}
@@ -38,10 +56,10 @@ class PikabuBridge extends BridgeAbstract {
}
public function getName() {
if (is_string($this->getInput('tag'))) {
return $this->getInput('tag') . ' - ' . parent::getName();
} else {
if (is_null($this->title)) {
return parent::getName();
} else {
return $this->title . ' - ' . parent::getName();
}
}
@@ -52,6 +70,8 @@ class PikabuBridge extends BridgeAbstract {
$text_html = iconv('windows-1251', 'utf-8', $text_html);
$html = str_get_html($text_html);
$this->title = $html->find('title', 0)->innertext;
foreach($html->find('article.story') as $post) {
$time = $post->find('time.story__datetime', 0);
if (is_null($time)) continue;
@@ -67,6 +87,11 @@ class PikabuBridge extends BridgeAbstract {
}
}
foreach($post->find('[data-type=gifx]') as $el) {
$src = $el->getAttribute('data-source');
$el->outertext = '<img src="' . $src . '">';
}
foreach($post->find('img') as $img) {
$src = $img->getAttribute('src');
if (!$src) {

132
bridges/QPlayBridge.php Normal file
View File

@@ -0,0 +1,132 @@
<?php
class QPlayBridge extends BridgeAbstract {
const NAME = 'Q Play';
const URI = 'https://www.qplay.pt';
const DESCRIPTION = 'Entretenimento e humor em Português';
const MAINTAINER = 'somini';
const PARAMETERS = array(
'Program' => array(
'program' => array(
'name' => 'Program Name',
'type' => 'text',
'required' => true,
),
),
'Catalog' => array(
'all_pages' => array(
'name' => 'All Pages',
'type' => 'checkbox',
'defaultValue' => false,
),
),
);
public function getIcon() {
# This should be the favicon served on `self::URI`
return 'https://s3.amazonaws.com/unode1/assets/4957/r3T9Lm9LTLmpAEX6FlSA_apple-touch-icon.png';
}
public function getURI() {
switch ($this->queriedContext) {
case 'Program':
return self::URI . '/programs/' . $this->getInput('program');
case 'Catalog':
return self::URI . '/catalog';
}
return parent::getURI();
}
public function getName() {
switch ($this->queriedContext) {
case 'Program':
$html = getSimpleHTMLDOMCached($this->getURI())
or returnServerError('Could not load content');
return $html->find('h1.program--title', 0)->innertext;
case 'Catalog':
return self::NAME . ' | Programas';
}
return parent::getName();
}
/* This uses the uscreen platform, other sites can adapt this. https://www.uscreen.tv/ */
public function collectData() {
switch ($this->queriedContext) {
case 'Program':
$program = $this->getInput('program');
$html = getSimpleHTMLDOMCached($this->getURI())
or returnServerError('Could not load content');
foreach($html->find('.cce--thumbnails-video-chapter') as $element) {
$cid = $element->getAttribute('data-id');
$item['title'] = $element->find('.cce--chapter-title', 0)->innertext;
$item['content'] = $element->find('.cce--thumbnails-image-block', 0)
. $element->find('.cce--chapter-body', 0)->innertext;
$item['uri'] = $this->getURI() . '?cid=' . $cid;
/* TODO: Suport login credentials? */
/* # Get direct video URL */
/* $json_source = getContents(self::URI . '/chapters/' . $cid, array('Cookie: _uscreen2_session=???;')) */
/* or returnServerError('Could not request chapter JSON'); */
/* $json = json_decode($json_source); */
/* $item['enclosures'] = [$json->fallback]; */
$this->items[] = $item;
}
break;
case 'Catalog':
$json_raw = getContents($this->getCatalogURI(1))
or returnServerError('Could not load catalog content');
$json = json_decode($json_raw);
$total_pages = $json->total_pages;
foreach($this->parseCatalogPage($json) as $item) {
$this->items[] = $item;
}
if ($this->getInput('all_pages') === true) {
foreach(range(2, $total_pages) as $page) {
$json_raw = getContents($this->getCatalogURI($page))
or returnServerError('Could not load catalog content (all pages)');
$json = json_decode($json_raw);
foreach($this->parseCatalogPage($json) as $item) {
$this->items[] = $item;
}
}
}
break;
}
}
private function getCatalogURI($page) {
return self::URI . '/catalog.json?page=' . $page;
}
private function parseCatalogPage($json) {
$items = array();
foreach($json->records as $record) {
$item = array();
$item['title'] = $record->title;
$item['content'] = $record->description
. '<div>Duration: ' . $record->duration . '</div>';
$item['timestamp'] = strtotime($record->release_date);
$item['uri'] = self::URI . $record->url;
$item['enclosures'] = array(
$record->main_poster,
);
$items[] = $item;
}
return $items;
}
}

View File

@@ -1,34 +1,87 @@
<?php
class RadioMelodieBridge extends BridgeAbstract {
const NAME = 'Radio Melodie Actu';
const URI = 'https://www.radiomelodie.com/';
const URI = 'https://www.radiomelodie.com';
const DESCRIPTION = 'Retourne les actualités publiées par Radio Melodie';
const MAINTAINER = 'sysadminstory';
public function getIcon() {
return self::URI . 'img/favicon.png';
return self::URI . '/img/favicon.png';
}
public function collectData(){
$html = getSimpleHTMLDOM(self::URI . 'actu')
$html = getSimpleHTMLDOM(self::URI . '/actu/')
or returnServerError('Could not request Radio Melodie.');
$list = $html->find('div[class=actuitem]');
$list = $html->find('div[class=displayList]', 0)->children();
foreach($list as $element) {
$item = array();
if($element->tag == 'a') {
$articleURL = self::URI . $element->href;
$article = getSimpleHTMLDOM($articleURL);
$textDOM = $article->find('article', 0);
// Get picture URL
$pictureHTML = $element->find('div[class=picture]');
preg_match(
'/background-image:url\((.*)\);/',
$pictureHTML[0]->getAttribute('style'),
$pictures);
$pictureURL = $pictures[1];
// Initialise arrays
$item = array();
$audio = array();
$picture = array();
$item['enclosures'] = array($pictureURL);
$item['uri'] = self::URI . $element->parent()->href;
$item['title'] = $element->find('h3', 0)->plaintext;
$item['content'] = $element->find('p', 0)->plaintext . '<br/><img src="' . $pictureURL . '"/>';
$this->items[] = $item;
// Get the Main picture URL
$picture[] = $this->rewriteImage($article->find('div[id=pictureTitleSupport]', 0)->find('img', 0)->src);
$audioHTML = $article->find('audio');
// Add the audio element to the enclosure
foreach($audioHTML as $audioElement) {
$audioURL = $audioElement->src;
$audio[] = $audioURL;
}
// Rewrite pictures URL
$imgs = $textDOM->find('img[src^="http://www.radiomelodie.com/image.php]');
foreach($imgs as $img) {
$img->src = $this->rewriteImage($img->src);
$article->save();
}
// Remove Google Ads
$ads = $article->find('div[class=adInline]');
foreach($ads as $ad) {
$ad->outertext = '';
$article->save();
}
// Remove Radio Melodie Logo
$logoHTML = $article->find('div[id=logoArticleRM]', 0);
$logoHTML->outertext = '';
$article->save();
$author = $article->find('p[class=AuthorName]', 0)->plaintext;
$item['enclosures'] = array_merge($picture, $audio);
$item['author'] = $author;
$item['uri'] = $articleURL;
$item['title'] = $article->find('meta[property=og:title]', 0)->content;
$date = $article->find('p[class*=date]', 0)->plaintext;
// Header Image
$header = '<img src="' . $picture[0] . '"/>';
// Remove the Date and Author part
$textDOM->find('div[class=AuthorDate]', 0)->outertext = '';
$article->save();
$text = $textDOM->innertext;
$item['content'] = '<h1>' . $item['title'] . '</h1>' . $date . '<br/>' . $header . $text;
$this->items[] = $item;
}
}
}
/*
* Function to rewrite image URL to use the real Image URL and not the resized one (which is very slow)
*/
private function rewriteImage($url)
{
$parts = explode('?', $url);
parse_str(html_entity_decode($parts[1]), $params);
return self::URI . '/' . $params['image'];
}
}

View File

@@ -6,98 +6,60 @@ class RoadAndTrackBridge extends BridgeAbstract {
const CACHE_TIMEOUT = 86400; // 24h
const DESCRIPTION = 'Returns the latest news from Road & Track.';
const PARAMETERS = array(
array(
'new-cars' => array(
'name' => 'New Cars',
'type' => 'checkbox',
'exampleValue' => 'checked',
'title' => 'Activate to load New Cars articles'
),
'motorsports' => array(
'name' => 'Motorsports',
'type' => 'checkbox',
'exampleValue' => 'checked',
'title' => 'Activate to load Motorsports articles'
),
'car-culture' => array(
'name' => 'Car Culture',
'type' => 'checkbox',
'exampleValue' => 'checked',
'title' => 'Activate to load Car Culture articles'
),
'car-shows' => array(
'name' => 'Car shows',
'type' => 'checkbox',
'exampleValue' => 'checked',
'title' => 'Activate to load Car shows articles'
)
)
);
const SIG_URL = 'https://cloud.mazdigital.com/feeds/production/comboapp/204/api/v3/';
public function collectData() {
//Magic
$signVal = '?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9jbG91ZC5tYXpkaWd';
$signVal .= 'pdGFsLmNvbS9mZWVkcy9wcm9kdWN0aW9uL2NvbWJvYXBwLzIwNC8qIiwiQ29uZGl0aW9uIj';
$signVal .= 'p7IkRhdGVMZXNzVGhhbiI6eyJBV1M6RXBvY2hUaW1lIjoxNTUyNTU5MDUzfSwiSXBBZGRyZ';
$signVal .= 'XNzIjp7IkFXUzpTb3VyY2VJcCI6IjAuMC4wLjAvMCJ9fX1dfQ__&Signature=jgS~Jccjs';
$signVal .= 'lXMMywWesmwDpUbHvEmrADRP7iBRzT~OiP-O~zI-8TtQzqTP7GUrpB9~v69CvhO7-JVtw94';
$signVal .= 'VC3N6lQrwsxTTIhpS57YGeV~MbZx~P653yUV7jb3jpJE2yUawfXnEkD-XzOIn8-caMo~14i';
$signVal .= 'KuWV9KNDkTJaRgOMy0rrVpWqiuBjCu5s5B8Ylt2qwcpOvHjXSqG9IY5c7GUIXKsk8yXzGFi';
$signVal .= 'yzy8hfuGgdx0n7fgl7c4-EoDgQaz~U76g0epejPxV5Csj16rCCfAqBU5kZJnACZ1vvOvRcV';
$signVal .= 'Wiu8KUuUuCS04SPmJ73Y5XoY8~uXRScxZG1kAFTIAhT4nYVlg__&Key-Pair-Id=APKAIZB';
$signVal .= 'QNNSW4WGIFP4Q';
$page = getSimpleHTMLDOM(self::URI);
$newsElements = array();
if($this->getInput('new-cars')) {
$newsElements = array_merge($newsElements,
json_decode(getContents(self::SIG_URL . '7591/item_feed' . $signVal))
);
}
if($this->getInput('motorsports')) {
$newsElements = array_merge($newsElements,
json_decode(getContents(self::SIG_URL . '7590/item_feed' . $signVal))
);
}
if($this->getInput('car-culture')) {
$newsElements = array_merge($newsElements,
json_decode(getContents(self::SIG_URL . '7588/item_feed' . $signVal))
);
}
if($this->getInput('car-shows')) {
$newsElements = array_merge($newsElements,
json_decode(getContents(self::SIG_URL . '7589/item_feed' . $signVal))
);
}
usort($newsElements, function($a, $b) {
return $b->published - $a->published;
});
//Process the first element
$firstArticleLink = $page->find('.custom-promo-title', 0)->href;
$this->items[] = $this->fetchArticle($firstArticleLink);
$limit = 19;
foreach($newsElements as $element) {
$item = array();
$item['uri'] = $element->sourceUrl;
$item['timestamp'] = $element->published;
$item['enclosures'] = array($element->cover->url);
$item['title'] = $element->title;
$item['content'] = $this->getArticleContent($element);
$this->items[] = $item;
if($limit > 0) {
$limit--;
} else {
break;
}
foreach($page->find('.full-item-title') as $article) {
$this->items[] = $this->fetchArticle($article->href);
$limit -= 1;
if($limit == 0) break;
}
}
private function fixImages($content) {
$enclosures = [];
foreach($content->find('img') as $image) {
$image->src = explode('?', $image->getAttribute('data-src'))[0];
$enclosures[] = $image->src;
}
foreach($content->find('.embed-image-wrap, .content-lede-image-wrap') as $imgContainer) {
$imgContainer->style = '';
}
return $enclosures;
}
private function fetchArticle($articleLink) {
$articleLink = self::URI . $articleLink;
$article = getSimpleHTMLDOM($articleLink);
$item = array();
$item['title'] = $article->find('.content-hed', 0)->innertext;
$item['author'] = $article->find('.byline-name', 0)->innertext;
$item['timestamp'] = strtotime($article->find('.content-info-date', 0)->getAttribute('datetime'));
$content = $article->find('.content-container', 0);
if($content->find('.content-rail', 0) !== null)
$content->find('.content-rail', 0)->innertext = '';
$enclosures = $this->fixImages($content);
$item['enclosures'] = $enclosures;
$item['content'] = $content;
return $item;
}
private function getArticleContent($article) {
return getContents($article->contentUrl);

View File

@@ -9,7 +9,7 @@ class Rue89Bridge extends BridgeAbstract {
public function collectData() {
$jsonArticles = getContents('https://appdata.nouvelobs.com/rue89/feed.json')
or die('Unable to query Rue89 !');
or returnServerError('Unable to query Rue89 !');
$articles = json_decode($jsonArticles)->items;
foreach($articles as $article) {
$this->items[] = $this->getArticle($article);
@@ -19,7 +19,8 @@ class Rue89Bridge extends BridgeAbstract {
private function getArticle($articleInfo) {
$articleJson = getContents($articleInfo->json_url) or die('Unable to get article !');
$articleJson = getContents($articleInfo->json_url)
or returnServerError('Unable to get article !');
$article = json_decode($articleJson);
$item = array();
$item['title'] = $article->title;

63
bridges/SIMARBridge.php Normal file
View File

@@ -0,0 +1,63 @@
<?php
class SIMARBridge extends BridgeAbstract {
const NAME = 'SIMAR';
const URI = 'http://www.simar-louresodivelas.pt/';
const DESCRIPTION = 'Verificar estado da rede SIMAR';
const MAINTAINER = 'somini';
const PARAMETERS = array(
'Público' => array(
'interventions' => array(
'type' => 'checkbox',
'name' => 'Incluir Intervenções?',
'defaultValue' => 'checked',
)
)
);
public function collectData() {
$html = getSimpleHTMLDOM(self::getURI())
or returnServerError('Could not load content');
$e_home = $html->find('#home', 0)
or returnServerError('Invalid site structure');
foreach($e_home->find('span') as $element) {
$item = array();
$item['title'] = 'Rotura: ' . $element->plaintext;
$item['content'] = $element->innertext;
$item['uid'] = 'urn:sha1:' . hash('sha1', $item['content']);
$this->items[] = $item;
}
if ($this->getInput('interventions')) {
$e_main1 = $html->find('#menu1', 0)
or returnServerError('Invalid site structure');
foreach ($e_main1->find('a') as $element) {
$item = array();
$item['title'] = 'Intervenção: ' . $element->plaintext;
$item['uri'] = self::getURI() . $element->href;
$item['content'] = $element->innertext;
/* Try to get the actual contents for this kind of item */
$item_html = getSimpleHTMLDOMCached($item['uri']);
if ($item_html) {
$e_item = $item_html->find('.auto-style59', 0);
foreach($e_item->find('p') as $paragraph) {
/* Remove empty paragraphs */
if (preg_match('/^(\W|&nbsp;)+$/', $paragraph->innertext) == 1) {
$paragraph->outertext = '';
}
}
if ($e_item) {
$item['content'] = $e_item->innertext;
}
}
$this->items[] = $item;
}
}
}
}

View File

@@ -16,6 +16,8 @@ class SoundCloudBridge extends BridgeAbstract {
const CLIENT_ID = 'W0KEWWILAjDiRH89X0jpwzuq6rbSK08R';
private $feedIcon = null;
public function collectData(){
$res = json_decode(getContents(
@@ -25,6 +27,8 @@ class SoundCloudBridge extends BridgeAbstract {
. self::CLIENT_ID
)) or returnServerError('No results for this query');
$this->feedIcon = $res->avatar_url;
$tracks = json_decode(getContents(
'https://api.soundcloud.com/users/'
. urlencode($res->id)
@@ -56,6 +60,14 @@ class SoundCloudBridge extends BridgeAbstract {
}
public function getIcon(){
if ($this->feedIcon) {
return $this->feedIcon;
}
return parent::getIcon();
}
public function getName(){
if(!is_null($this->getInput('u'))) {
return self::NAME . ' - ' . $this->getInput('u');

View File

@@ -8,44 +8,12 @@ class SteamBridge extends BridgeAbstract {
const MAINTAINER = 'jacknumber';
const PARAMETERS = array(
'Wishlist' => array(
'username' => array(
'name' => 'Username',
'userid' => array(
'name' => 'Steamid64 (find it on steamid.io)',
'title' => 'User ID (17 digits). Find your user ID with steamid.io or steamidfinder.com',
'required' => true,
),
'currency' => array(
'name' => 'Currency',
'type' => 'list',
'values' => array(
// source: http://steam.steamlytics.xyz/currencies
'USD' => 'us',
'GBP' => 'gb',
'EUR' => 'fr',
'CHF' => 'ch',
'RUB' => 'ru',
'BRL' => 'br',
'JPY' => 'jp',
'SEK' => 'se',
'IDR' => 'id',
'MYR' => 'my',
'PHP' => 'ph',
'SGD' => 'sg',
'THB' => 'th',
'KRW' => 'kr',
'TRY' => 'tr',
'MXN' => 'mx',
'CAD' => 'ca',
'NZD' => 'nz',
'CNY' => 'cn',
'INR' => 'in',
'CLP' => 'cl',
'PEN' => 'pe',
'COP' => 'co',
'ZAR' => 'za',
'HKD' => 'hk',
'TWD' => 'tw',
'SRD' => 'sr',
'AED' => 'ae',
),
'exampleValue' => '76561198821231205',
'pattern' => '[0-9]{17}',
),
'only_discount' => array(
'name' => 'Only discount',
@@ -56,27 +24,15 @@ class SteamBridge extends BridgeAbstract {
public function collectData(){
$username = $this->getInput('username');
$params = array(
'cc' => $this->getInput('currency')
);
$userid = $this->getInput('userid');
$url = self::URI . 'wishlist/id/' . $username . '?' . http_build_query($params);
$targetVariable = 'g_rgAppInfo';
$sourceUrl = self::URI . 'wishlist/profiles/' . $userid . '/wishlistdata?p=0';
$sort = array();
$html = '';
$html = getSimpleHTMLDOM($url)
or returnServerError("Could not request Steam Wishlist. Tried:\n - $url");
$json = getContents($sourceUrl)
or returnServerError('Could not get content from wishlistdata (' . $sourceUrl . ')');
$jsContent = $html->find('.responsive_page_template_content script', 0)->innertext;
if(preg_match('/var ' . $targetVariable . ' = (.*?);/s', $jsContent, $matches)) {
$appsData = json_decode($matches[1]);
} else {
returnServerError("Could not parse JS variable ($targetVariable) in page content.");
}
$appsData = json_decode($json);
foreach($appsData as $id => $element) {
@@ -87,6 +43,8 @@ class SteamBridge extends BridgeAbstract {
if($element->subs) {
$appIsBuyable = 1;
$priceBlock = str_get_html($element->subs[0]->discount_block);
$appPrice = str_replace('--', '00', $priceBlock->find('.discount_final_price', 0)->plaintext);
if($element->subs[0]->discount_pct) {
@@ -94,8 +52,6 @@ class SteamBridge extends BridgeAbstract {
$discountBlock = str_get_html($element->subs[0]->discount_block);
$appDiscountValue = $discountBlock->find('.discount_pct', 0)->plaintext;
$appOldPrice = $discountBlock->find('.discount_original_price', 0)->plaintext;
$appNewPrice = $discountBlock->find('.discount_final_price', 0)->plaintext;
$appPrice = $appNewPrice;
} else {
@@ -103,7 +59,6 @@ class SteamBridge extends BridgeAbstract {
continue;
}
$appPrice = $element->subs[0]->price / 100;
}
} else {
@@ -117,11 +72,14 @@ class SteamBridge extends BridgeAbstract {
}
}
$coverUrl = str_replace('_292x136', '', strtok($element->capsule, '?'));
$picturesPath = pathinfo($coverUrl)['dirname'] . '/';
$item = array();
$item['uri'] = "http://store.steampowered.com/app/$id/";
$item['title'] = $element->name;
$item['type'] = $appType;
$item['cover'] = str_replace('_292x136', '', $element->capsule);
$item['cover'] = $coverUrl;
$item['timestamp'] = $element->added;
$item['isBuyable'] = $appIsBuyable;
$item['hasDiscount'] = $appHasDiscount;
@@ -129,22 +87,29 @@ class SteamBridge extends BridgeAbstract {
$item['priority'] = $element->priority;
if($appIsBuyable) {
$item['price'] = floatval(str_replace(',', '.', $appPrice));
$item['content'] = $appPrice;
}
if($appIsFree) {
$item['content'] = 'Free';
}
if($appHasDiscount) {
$item['discount']['value'] = $appDiscountValue;
$item['discount']['oldPrice'] = floatval(str_replace(',', '.', $appOldPrice));
$item['discount']['newPrice'] = floatval(str_replace(',', '.', $appNewPrice));
$item['discount']['oldPrice'] = $appOldPrice;
$item['content'] = '<s>' . $appOldPrice . '</s> <b>' . $appPrice . '</b> (' . $appDiscountValue . ')';
}
$item['enclosures'] = array();
$item['enclosures'][] = str_replace('_292x136', '', $element->capsule);
$item['enclosures'][] = $coverUrl;
foreach($element->screenshots as $screenshot) {
$item['enclosures'][] = substr($element->capsule, 0, -31) . $screenshot;
foreach($element->screenshots as $screenshotFileName) {
$item['enclosures'][] = $picturesPath . $screenshotFileName;
}
$sort[$id] = $element->priority;

View File

@@ -0,0 +1,127 @@
<?php
class SteamCommunityBridge extends BridgeAbstract {
const NAME = 'Steam Community';
const URI = 'https://www.steamcommunity.com';
const DESCRIPTION = 'Get the latest community updates for a game on Steam.';
const MAINTAINER = 'thefranke';
const CACHE_TIMEOUT = 3600; // 1h
const PARAMETERS = array(
array(
'i' => array(
'name' => 'App ID',
'required' => true
),
'category' => array(
'name' => 'category',
'type' => 'list',
'exampleValue' => 'Artwork',
'title' => 'Select a category',
'values' => array(
'Artwork' => 'images',
'Screenshots' => 'screenshots',
'Videos' => 'videos'
)
)
)
);
public function getIcon() {
return self::URI . '/favicon.ico';
}
protected function getMainPage() {
$category = $this->getInput('category');
$html = getSimpleHTMLDOM($this->getURI() . '/?p=1&browsefilter=mostrecent')
or returnServerError('Could not fetch Steam data.');
return $html;
}
public function getName() {
$category = $this->getInput('category');
if (is_null('i') || is_null($category)) {
return self::NAME;
}
$html = $this->getMainPage();
$titleItem = $html->find('div.apphub_AppName', 0);
if (!$titleItem)
return self::NAME;
return $titleItem->innertext . ' (' . ucwords($category) . ')';
}
public function getURI() {
return self::URI . '/app/'
. $this->getInput('i') . '/'
. $this->getInput('category');
}
public function collectData() {
$category = $this->getInput('category');
$html = $this->getMainPage();
$cards = $html->find('div.apphub_Card');
foreach($cards as $card) {
$uri = $card->getAttribute('data-modal-content-url');
$htmlCard = getSimpleHTMLDOMCached($uri);
$author = $card->find('div.apphub_CardContentAuthorName', 0)->innertext;
$author = strip_tags($author);
$title = $author . '\'s screenshot';
if ($category != 'screenshots')
$title = $htmlCard->find('div.workshopItemTitle', 0)->innertext;
$date = $htmlCard->find('div.detailsStatRight', 0)->innertext;
// create item
$item = array();
$item['title'] = $title;
$item['uri'] = $uri;
$item['timestamp'] = strtotime($date);
$item['author'] = $author;
$item['categories'] = $category;
$media = $htmlCard->getElementById('ActualMedia');
$mediaURI = $media->getAttribute('src');
$downloadURI = $mediaURI;
if ($category == 'videos') {
preg_match('/.*\/embed\/(.*)\?/', $mediaURI, $result);
$youtubeID = $result[1];
$mediaURI = 'https://img.youtube.com/vi/' . $youtubeID . '/hqdefault.jpg';
$downloadURI = 'https://www.youtube.com/watch?v=' . $youtubeID;
}
$desc = '';
if ($category == 'screenshots') {
$descItem = $htmlCard->find('div.screenshotDescription', 0);
if ($descItem)
$desc = $descItem->innertext;
}
if ($category == 'images') {
$descItem = $htmlCard->find('div.nonScreenshotDescription', 0);
if ($descItem)
$desc = $descItem->innertext;
$downloadURI = $htmlCard->find('a.downloadImage', 0)->href;
}
$item['content'] = '<p><a href="' . $downloadURI . '"><img src="' . $mediaURI . '"/></a></p>';
$item['content'] .= '<p>' . $desc . '</p>';
$this->items[] = $item;
if (count($this->items) >= 10)
break;
}
}
}

View File

@@ -0,0 +1,80 @@
<?php
class StockFilingsBridge extends FeedExpander {
const MAINTAINER = 'captn3m0';
const NAME = 'SEC Stock filings';
const URI = 'https://www.sec.gov/edgar/searchedgar/companysearch.html';
const CACHE_TIMEOUT = 3600; // 1h
const DESCRIPTION = 'Tracks SEC Filings for a single company';
const SEARCH_URL = 'https://www.sec.gov/cgi-bin/browse-edgar?owner=exclude&action=getcompany&CIK=';
const WEBSITE_ROOT = 'https://www.sec.gov';
const PARAMETERS = array(
array(
'ticker' => array(
'name' => 'cik',
'required' => true,
'exampleValue' => 'AMD',
// https://stackoverflow.com/a/12827734
'pattern' => '[A-Za-z0-9]+',
),
));
public function getIcon() {
return 'https://www.sec.gov/favicon.ico';
}
/**
* Generates search URL
*/
private function getSearchUrl() {
return self::SEARCH_URL . $this->getInput('ticker');
}
/**
* Returns the Company Name
*/
private function getRssFeed($html) {
$links = $html->find('#contentDiv a');
foreach ($links as $link) {
$href = $link->href;
if (substr($href, 0, 4) !== 'http') {
$href = self::WEBSITE_ROOT . $href;
}
parse_str(html_entity_decode(parse_url($href, PHP_URL_QUERY)), $query);
if (isset($query['output']) and ($query['output'] == 'atom')) {
return $href;
}
}
return false;
}
/**
* Return \simple_html_dom object
* for the entire html of the product page
*/
private function getHtml() {
$uri = $this->getSearchUrl();
return getSimpleHTMLDOM($uri) ?: returnServerError('Could not request SEC.');
}
/**
* Scrape the SEC Stock Filings RSS Feed URL
* and redirect there
*/
public function collectData() {
$html = $this->getHtml();
$rssFeedUrl = $this->getRssFeed($html);
if ($rssFeedUrl) {
parent::collectExpandableDatas($rssFeedUrl);
} else {
returnClientError('Could not find RSS Feed URL. Are you sure you used a valid CIK?');
}
}
}

View File

@@ -13,6 +13,10 @@ class VkBridge extends BridgeAbstract
'u' => array(
'name' => 'Group or user name',
'required' => true
),
'hide_reposts' => array(
'name' => 'Hide reposts',
'type' => 'checkbox',
)
)
);
@@ -48,7 +52,7 @@ class VkBridge extends BridgeAbstract
$text_html = $this->getContents()
or returnServerError('No results for group or user name "' . $this->getInput('u') . '".');
$text_html = iconv('windows-1251', 'utf-8', $text_html);
$text_html = iconv('windows-1251', 'utf-8//ignore', $text_html);
// makes album link generating work correctly
$text_html = str_replace('"class="page_album_link">', '" class="page_album_link">', $text_html);
$html = str_get_html($text_html);
@@ -234,6 +238,9 @@ class VkBridge extends BridgeAbstract
}
if (is_object($post->find('div.copy_quote', 0))) {
if ($this->getInput('hide_reposts') === true) {
continue;
}
$copy_quote = $post->find('div.copy_quote', 0);
if ($copy_post_header = $copy_quote->find('div.copy_post_header', 0)) {
$copy_post_header->outertext = '';

View File

@@ -71,16 +71,4 @@ class WordPressPluginUpdateBridge extends BridgeAbstract {
return parent::getName();
}
private function getCachedDate($url){
Debug::log('getting pubdate from url ' . $url . '');
// Initialize cache
$cache = Cache::create(Configuration::getConfig('cache', 'type'));
$cache->setPath(PATH_CACHE . 'pages/');
$params = [$url];
$cache->setParameters($params);
// Get cachefile timestamp
$time = $cache->getTime();
return ($time !== false ? $time : time());
}
}

View File

@@ -229,7 +229,7 @@ class YoutubeBridge extends BridgeAbstract {
$url_listing = self::URI . 'playlist?list=' . urlencode($this->request);
$html = $this->ytGetSimpleHTMLDOM($url_listing)
or returnServerError("Could not request YouTube. Tried:\n - $url_listing");
$item_count = $this->ytBridgeParseHtmlListing($html, 'tr.pl-video', '.pl-video-title a', false);
$item_count = $this->ytBridgeParseHtmlListing($html, 'tr.pl-video', '.pl-video-title a', true);
if ($item_count <= 15 && !$this->skipFeeds() && ($xml = $this->ytGetSimpleHTMLDOM($url_feed))) {
$this->ytBridgeParseXmlFeed($xml);
} else {

View File

@@ -3,20 +3,21 @@
* Cache with file system
*/
class FileCache implements CacheInterface {
protected $path;
protected $param;
protected $key;
public function loadData(){
if(file_exists($this->getCacheFile())) {
return unserialize(file_get_contents($this->getCacheFile()));
}
return null;
}
public function saveData($datas){
public function saveData($data){
// Notice: We use plain serialize() here to reduce memory footprint on
// large input data.
$writeStream = file_put_contents($this->getCacheFile(), serialize($datas));
$writeStream = file_put_contents($this->getCacheFile(), serialize($data));
if($writeStream === false) {
throw new \Exception('Cannot write the cache... Do you have the right permissions ?');
@@ -29,13 +30,14 @@ class FileCache implements CacheInterface {
$cacheFile = $this->getCacheFile();
clearstatcache(false, $cacheFile);
if(file_exists($cacheFile)) {
return filemtime($cacheFile);
$time = filemtime($cacheFile);
return ($time !== false) ? $time : null;
}
return false;
return null;
}
public function purgeCache($duration){
public function purgeCache($seconds){
$cachePath = $this->getPath();
if(file_exists($cachePath)) {
$cacheIterator = new RecursiveIteratorIterator(
@@ -47,7 +49,7 @@ class FileCache implements CacheInterface {
if(in_array($cacheFile->getBasename(), array('.', '..', '.gitkeep')))
continue;
elseif($cacheFile->isFile()) {
if(filemtime($cacheFile->getPathname()) < time() - $duration)
if(filemtime($cacheFile->getPathname()) < time() - $seconds)
unlink($cacheFile->getPathname());
}
}
@@ -55,34 +57,34 @@ class FileCache implements CacheInterface {
}
/**
* Set cache path
* Set scope
* @return self
*/
public function setPath($path){
if(is_null($path) || !is_string($path)) {
throw new \Exception('The given path is invalid!');
public function setScope($scope){
if(is_null($scope) || !is_string($scope)) {
throw new \Exception('The given scope is invalid!');
}
$this->path = $path;
// Make sure path ends with '/' or '\'
$lastchar = substr($this->path, -1, 1);
if($lastchar !== '/' && $lastchar !== '\\')
$this->path .= '/';
if(!is_dir($this->path))
mkdir($this->path, 0755, true);
$this->path = PATH_CACHE . trim($scope, " \t\n\r\0\x0B\\\/") . '/';
return $this;
}
/**
* Set HTTP GET parameters
* Set key
* @return self
*/
public function setParameters(array $param){
$this->param = array_map('strtolower', $param);
public function setKey($key){
if (!empty($key) && is_array($key)) {
$key = array_map('strtolower', $key);
}
$key = json_encode($key);
if (!is_string($key)) {
throw new \Exception('The given key is invalid!');
}
$this->key = $key;
return $this;
}
@@ -90,9 +92,15 @@ class FileCache implements CacheInterface {
* Return cache path (and create if not exist)
* @return string Cache path
*/
protected function getPath(){
private function getPath(){
if(is_null($this->path)) {
throw new \Exception('Call "setPath" first!');
throw new \Exception('Call "setScope" first!');
}
if(!is_dir($this->path)) {
if (mkdir($this->path, 0755, true) !== true) {
throw new \Exception('Unable to create ' . $this->path);
}
}
return $this->path;
@@ -102,7 +110,7 @@ class FileCache implements CacheInterface {
* Get the file name use for cache store
* @return string Path to the file cache
*/
protected function getCacheFile(){
private function getCacheFile(){
return $this->getPath() . $this->getCacheName();
}
@@ -110,13 +118,11 @@ class FileCache implements CacheInterface {
* Determines file name for store the cache
* return string
*/
protected function getCacheName(){
if(is_null($this->param)) {
throw new \Exception('Call "setParameters" first!');
private function getCacheName(){
if(is_null($this->key)) {
throw new \Exception('Call "setKey" first!');
}
// Change character when making incompatible changes to prevent loading
// errors due to incompatible file contents \|/
return hash('md5', http_build_query($this->param) . 'A') . '.cache';
return hash('md5', $this->key) . '.cache';
}
}

115
caches/MemcachedCache.php Normal file
View File

@@ -0,0 +1,115 @@
<?php
class MemcachedCache implements CacheInterface {
private $scope;
private $key;
private $conn;
private $expiration = 0;
private $time = false;
private $data = null;
public function __construct() {
if (!extension_loaded('memcached')) {
returnServerError('"memcached" extension not loaded. Please check "php.ini"');
}
$host = Configuration::getConfig(get_called_class(), 'host');
$port = Configuration::getConfig(get_called_class(), 'port');
if (empty($host) && empty($port)) {
returnServerError('Configuration for ' . get_called_class() . ' missing. Please check your ' . FILE_CONFIG);
} else if (empty($host)) {
returnServerError('"host" param is not set for ' . get_called_class() . '. Please check your ' . FILE_CONFIG);
} else if (empty($port)) {
returnServerError('"port" param is not set for ' . get_called_class() . '. Please check your ' . FILE_CONFIG);
} else if (!ctype_digit($port)) {
returnServerError('"port" param is invalid for ' . get_called_class() . '. Please check your ' . FILE_CONFIG);
}
$port = intval($port);
if ($port < 1 || $port > 65535) {
returnServerError('"port" param is invalid for ' . get_called_class() . '. Please check your ' . FILE_CONFIG);
}
$conn = new Memcached();
$conn->addServer($host, $port) or returnServerError('Could not connect to memcached server');
$this->conn = $conn;
}
public function loadData(){
if ($this->data) return $this->data;
$result = $this->conn->get($this->getCacheKey());
if ($result === false) {
return false;
}
$this->time = $result['time'];
$this->data = $result['data'];
return $result['data'];
}
public function saveData($datas){
$time = time();
$object_to_save = array(
'data' => $datas,
'time' => $time,
);
$result = $this->conn->set($this->getCacheKey(), $object_to_save, $this->expiration);
if($result === false) {
returnServerError('Cannot write the cache to memcached server');
}
$this->time = $time;
return $this;
}
public function getTime(){
if ($this->time === false) {
$this->loadData();
}
return $this->time;
}
public function purgeCache($duration){
// Note: does not purges cache right now
// Just sets cache expiration and leave cache purging for memcached itself
$this->expiration = $duration;
}
/**
* Set scope
* @return self
*/
public function setScope($scope){
$this->scope = $scope;
return $this;
}
/**
* Set key
* @return self
*/
public function setKey($key){
if (!empty($key) && is_array($key)) {
$key = array_map('strtolower', $key);
}
$key = json_encode($key);
if (!is_string($key)) {
throw new \Exception('The given key is invalid!');
}
$this->key = $key;
return $this;
}
private function getCacheKey(){
if(is_null($this->key)) {
returnServerError('Call "setKey" first!');
}
return 'rss_bridge_cache_' . hash('md5', $this->scope . $this->key . 'A');
}
}

View File

@@ -3,16 +3,25 @@
* Cache based on SQLite 3 <https://www.sqlite.org>
*/
class SQLiteCache implements CacheInterface {
protected $path;
protected $param;
protected $scope;
protected $key;
private $db = null;
public function __construct() {
if (!extension_loaded('sqlite3'))
if (!extension_loaded('sqlite3')) {
die('"sqlite3" extension not loaded. Please check "php.ini"');
}
$file = PATH_CACHE . 'cache.sqlite';
$file = Configuration::getConfig(get_called_class(), 'file');
if (empty($file)) {
die('Configuration for ' . get_called_class() . ' missing. Please check your ' . FILE_CONFIG);
}
if (dirname($file) == '.') {
$file = PATH_CACHE . $file;
} elseif (!is_dir(dirname($file))) {
die('Invalid configuration for ' . get_called_class() . '. Please check your ' . FILE_CONFIG);
}
if (!is_file($file)) {
$this->db = new SQLite3($file);
@@ -39,10 +48,10 @@ class SQLiteCache implements CacheInterface {
return null;
}
public function saveData($datas){
public function saveData($data){
$Qupdate = $this->db->prepare('INSERT OR REPLACE INTO storage (key, value, updated) VALUES (:key, :value, :updated)');
$Qupdate->bindValue(':key', $this->getCacheKey());
$Qupdate->bindValue(':value', serialize($datas));
$Qupdate->bindValue(':value', serialize($data));
$Qupdate->bindValue(':updated', time());
$Qupdate->execute();
@@ -60,40 +69,53 @@ class SQLiteCache implements CacheInterface {
}
}
return false;
return null;
}
public function purgeCache($duration){
public function purgeCache($seconds){
$Qdelete = $this->db->prepare('DELETE FROM storage WHERE updated < :expired');
$Qdelete->bindValue(':expired', time() - $duration);
$Qdelete->bindValue(':expired', time() - $seconds);
$Qdelete->execute();
}
/**
* Set cache path
* Set scope
* @return self
*/
public function setPath($path){
$this->path = $path;
public function setScope($scope){
if(is_null($scope) || !is_string($scope)) {
throw new \Exception('The given scope is invalid!');
}
$this->scope = $scope;
return $this;
}
/**
* Set HTTP GET parameters
* Set key
* @return self
*/
public function setParameters(array $param){
$this->param = array_map('strtolower', $param);
public function setKey($key){
if (!empty($key) && is_array($key)) {
$key = array_map('strtolower', $key);
}
$key = json_encode($key);
if (!is_string($key)) {
throw new \Exception('The given key is invalid!');
}
$this->key = $key;
return $this;
}
////////////////////////////////////////////////////////////////////////////
protected function getCacheKey(){
if(is_null($this->param)) {
throw new \Exception('Call "setParameters" first!');
private function getCacheKey(){
if(is_null($this->key)) {
throw new \Exception('Call "setKey" first!');
}
return hash('sha1', $this->path . http_build_query($this->param), true);
return hash('sha1', $this->scope . $this->key, true);
}
}

12
composer.json Normal file
View File

@@ -0,0 +1,12 @@
{
"require": {
"php": ">=5.6",
"ext-mbstring": "*",
"ext-sqlite3": "*",
"ext-curl": "*",
"ext-openssl": "*",
"ext-libxml": "*",
"ext-simplexml": "*",
"ext-json": "*"
}
}

26
composer.lock generated Normal file
View File

@@ -0,0 +1,26 @@
{
"_readme": [
"This file locks the dependencies of your project to a known state",
"Read more about it at https://getcomposer.org/doc/01-basic-usage.md#installing-dependencies",
"This file is @generated automatically"
],
"content-hash": "ef341ee18f28c7bd5832e188fe157734",
"packages": [],
"packages-dev": [],
"aliases": [],
"minimum-stability": "stable",
"stability-flags": [],
"prefer-stable": false,
"prefer-lowest": false,
"platform": {
"php": ">=5.6",
"ext-mbstring": "*",
"ext-sqlite3": "*",
"ext-curl": "*",
"ext-openssl": "*",
"ext-libxml": "*",
"ext-simplexml": "*",
"ext-json": "*"
},
"platform-dev": []
}

View File

@@ -4,6 +4,14 @@
; file, it will be replaced on the next update of RSS-Bridge! You can specify
; your own configuration in 'config.ini.php' (copy this file).
[system]
; Defines the timezone used by RSS-Bridge
; Find a list of supported timezones at
; https://www.php.net/manual/en/timezones.php
; timezone = "UTC" (default)
timezone = "UTC"
[cache]
; Defines the cache type used by RSS-Bridge
@@ -52,3 +60,12 @@ username = ""
; The password for authentication. Insert this password when prompted for login.
; Use a strong password to prevent others from guessing your login!
password = ""
; --- Cache specific configuration ---------------------------------------------
[SQLiteCache]
file = "cache.sqlite"
[MemcachedCache]
host = "localhost"
port = 11211

View File

@@ -6,8 +6,6 @@ Configuration::loadConfiguration();
Authentication::showPromptIfNeeded();
date_default_timezone_set('UTC');
/*
Move the CLI arguments to the $_GET array, in order to be able to use
rss-bridge from the command line
@@ -29,28 +27,8 @@ define('USER_AGENT',
ini_set('user_agent', USER_AGENT);
// default whitelist
$whitelist_default = array(
'BandcampBridge',
'CryptomeBridge',
'DansTonChatBridge',
'DuckDuckGoBridge',
'FacebookBridge',
'FlickrBridge',
'GooglePlusPostBridge',
'GoogleSearchBridge',
'IdenticaBridge',
'InstagramBridge',
'OpenClassroomsBridge',
'PinterestBridge',
'ScmbBridge',
'TwitterBridge',
'WikipediaBridge',
'YoutubeBridge');
try {
Bridge::setWhitelist($whitelist_default);
$actionFac = new \ActionFactory();
$actionFac->setWorkingDir(PATH_LIB_ACTIONS);

View File

@@ -192,7 +192,8 @@ class Bridge {
/**
* Returns the whitelist.
*
* On first call this function reads the whitelist from {@see WHITELIST}.
* On first call this function reads the whitelist from {@see WHITELIST} if
* the file exists, {@see WHITELIST_DEFAULT} otherwise.
* * Each line in the file specifies one bridge on the whitelist.
* * An empty file disables all bridges.
* * If the file only only contains `*`, all bridges are whitelisted.
@@ -210,19 +211,21 @@ class Bridge {
if($firstCall) {
// Create initial whitelist or load from disk
if (!file_exists(WHITELIST) && !empty(self::$whitelist)) {
file_put_contents(WHITELIST, implode("\n", self::$whitelist));
} elseif(file_exists(WHITELIST)) {
if(file_exists(WHITELIST)) {
$contents = trim(file_get_contents(WHITELIST));
} elseif(file_exists(WHITELIST_DEFAULT)) {
$contents = trim(file_get_contents(WHITELIST_DEFAULT));
} else {
$contents = '';
}
if($contents === '*') { // Whitelist all bridges
self::$whitelist = self::getBridgeNames();
} else {
self::$whitelist = array_map('self::sanitizeBridgeName', explode("\n", $contents));
if($contents === '*') { // Whitelist all bridges
self::$whitelist = self::getBridgeNames();
} else {
//self::$whitelist = array_map('self::sanitizeBridgeName', explode("\n", $contents));
foreach(explode("\n", $contents) as $bridgeName) {
self::$whitelist[] = self::sanitizeBridgeName($bridgeName);
}
}
}
@@ -280,6 +283,12 @@ class Bridge {
$name = $matches[1];
}
// Improve performance for correctly written bridge names
if(in_array($name, self::getBridgeNames())) {
$index = array_search($name, self::getBridgeNames());
return self::getBridgeNames()[$index];
}
// The name is valid if a corresponding bridge file is found on disk
if(in_array(strtolower($name), array_map('strtolower', self::getBridgeNames()))) {
$index = array_search(strtolower($name), array_map('strtolower', self::getBridgeNames()));

View File

@@ -13,38 +13,54 @@
/**
* The cache interface
*
* @todo Add missing function to the interface
* @todo Explain parameters and return values in more detail
* @todo Return self more often (to allow call chaining)
*/
interface CacheInterface {
/**
* Set scope of the current cache
*
* If $scope is an empty string, the cache is set to a global context.
*
* @param string $scope The scope the data is related to
*/
public function setScope($scope);
/**
* Set key to assign the current data
*
* Since $key can be anything, the cache implementation must ensure to
* assign the related data reliably; most commonly by serializing and
* hashing the key in an appropriate way.
*
* @param array $key The key the data is related to
*/
public function setKey($key);
/**
* Loads data from cache
*
* @return mixed The cache data
* @return mixed The cached data or null
*/
public function loadData();
/**
* Stores data to the cache
*
* @param mixed $datas The data to store
* @param mixed $data The data to store
* @return self The cache object
*/
public function saveData($datas);
public function saveData($data);
/**
* Returns the timestamp for the curent cache file
* Returns the timestamp for the curent cache data
*
* @return int Timestamp
* @return int Timestamp or null
*/
public function getTime();
/**
* Removes any data that is older than the specified duration from cache
* Removes any data that is older than the specified age from cache
*
* @param int $duration The cache duration in seconds
* @param int $seconds The cache age in seconds
*/
public function purgeCache($duration);
public function purgeCache($seconds);
}

View File

@@ -28,7 +28,7 @@ final class Configuration {
*
* @todo Replace this property by a constant.
*/
public static $VERSION = '2019-03-17';
public static $VERSION = '2019-06-08';
/**
* Holds the configuration data.
@@ -80,35 +80,31 @@ final class Configuration {
// Check PHP version
if(version_compare(PHP_VERSION, '5.6.0') === -1)
die('RSS-Bridge requires at least PHP version 5.6.0!');
self::reportError('RSS-Bridge requires at least PHP version 5.6.0!');
// extensions check
if(!extension_loaded('openssl'))
die('"openssl" extension not loaded. Please check "php.ini"');
self::reportError('"openssl" extension not loaded. Please check "php.ini"');
if(!extension_loaded('libxml'))
die('"libxml" extension not loaded. Please check "php.ini"');
self::reportError('"libxml" extension not loaded. Please check "php.ini"');
if(!extension_loaded('mbstring'))
die('"mbstring" extension not loaded. Please check "php.ini"');
self::reportError('"mbstring" extension not loaded. Please check "php.ini"');
if(!extension_loaded('simplexml'))
die('"simplexml" extension not loaded. Please check "php.ini"');
self::reportError('"simplexml" extension not loaded. Please check "php.ini"');
// Allow RSS-Bridge to run without curl module in CLI mode without root certificates
if(!extension_loaded('curl') && !(php_sapi_name() === 'cli' && empty(ini_get('curl.cainfo'))))
die('"curl" extension not loaded. Please check "php.ini"');
self::reportError('"curl" extension not loaded. Please check "php.ini"');
if(!extension_loaded('json'))
die('"json" extension not loaded. Please check "php.ini"');
self::reportError('"json" extension not loaded. Please check "php.ini"');
// Check cache folder permissions (write permissions required)
if(!is_writable(PATH_CACHE))
die('RSS-Bridge does not have write permissions for ' . PATH_CACHE . '!');
// Check whitelist file permissions
if(!file_exists(WHITELIST) && !is_writable(dirname(WHITELIST)))
die('RSS-Bridge does not have write permissions for ' . WHITELIST . '!');
self::reportError('RSS-Bridge does not have write permissions for ' . PATH_CACHE . '!');
}
@@ -118,15 +114,13 @@ final class Configuration {
* Returns an error message and aborts execution if the configuration is invalid.
*
* The RSS-Bridge configuration is split into two files:
* - `config.default.ini.php`: The default configuration file that ships with
* every release of RSS-Bridge (do not modify this file!).
* - `config.ini.php`: The local configuration file that can be modified by
* server administrators.
* - {@see FILE_CONFIG_DEFAULT} The default configuration file that ships
* with every release of RSS-Bridge (do not modify this file!).
* - {@see FILE_CONFIG} The local configuration file that can be modified
* by server administrators.
*
* The files must be located at {@see PATH_ROOT}
*
* RSS-Bridge will first load `config.default.ini.php` into memory and then
* replace parameters with the contents of `config.ini.php`. That way new
* RSS-Bridge will first load {@see FILE_CONFIG_DEFAULT} into memory and then
* replace parameters with the contents of {@see FILE_CONFIG}. That way new
* parameters are automatically initialized with default values and custom
* configurations can be reduced to the minimum set of parametes necessary
* (only the ones that changed).
@@ -140,16 +134,16 @@ final class Configuration {
*/
public static function loadConfiguration() {
if(!file_exists(PATH_ROOT . 'config.default.ini.php'))
die('The default configuration file "config.default.ini.php" is missing!');
if(!file_exists(FILE_CONFIG_DEFAULT))
self::reportError('The default configuration file is missing at ' . FILE_CONFIG_DEFAULT);
Configuration::$config = parse_ini_file(PATH_ROOT . 'config.default.ini.php', true, INI_SCANNER_TYPED);
Configuration::$config = parse_ini_file(FILE_CONFIG_DEFAULT, true, INI_SCANNER_TYPED);
if(!Configuration::$config)
die('Error parsing config.default.ini.php');
self::reportError('Error parsing ' . FILE_CONFIG_DEFAULT);
if(file_exists(PATH_ROOT . 'config.ini.php')) {
if(file_exists(FILE_CONFIG)) {
// Replace default configuration with custom settings
foreach(parse_ini_file(PATH_ROOT . 'config.ini.php', true, INI_SCANNER_TYPED) as $header => $section) {
foreach(parse_ini_file(FILE_CONFIG, true, INI_SCANNER_TYPED) as $header => $section) {
foreach($section as $key => $value) {
// Skip unknown sections and keys
if(array_key_exists($header, Configuration::$config) && array_key_exists($key, Configuration::$config[$header])) {
@@ -159,8 +153,14 @@ final class Configuration {
}
}
if(!is_string(self::getConfig('system', 'timezone'))
|| !in_array(self::getConfig('system', 'timezone'), timezone_identifiers_list(DateTimeZone::ALL_WITH_BC)))
self::reportConfigurationError('system', 'timezone');
date_default_timezone_set(self::getConfig('system', 'timezone'));
if(!is_string(self::getConfig('proxy', 'url')))
die('Parameter [proxy] => "url" is not a valid string! Please check "config.ini.php"!');
self::reportConfigurationError('proxy', 'url', 'Is not a valid string');
if(!empty(self::getConfig('proxy', 'url'))) {
/** URL of the proxy server */
@@ -168,38 +168,38 @@ final class Configuration {
}
if(!is_bool(self::getConfig('proxy', 'by_bridge')))
die('Parameter [proxy] => "by_bridge" is not a valid Boolean! Please check "config.ini.php"!');
self::reportConfigurationError('proxy', 'by_bridge', 'Is not a valid Boolean');
/** True if proxy usage can be enabled selectively for each bridge */
define('PROXY_BYBRIDGE', self::getConfig('proxy', 'by_bridge'));
if(!is_string(self::getConfig('proxy', 'name')))
die('Parameter [proxy] => "name" is not a valid string! Please check "config.ini.php"!');
self::reportConfigurationError('proxy', 'name', 'Is not a valid string');
/** Name of the proxy server */
define('PROXY_NAME', self::getConfig('proxy', 'name'));
if(!is_string(self::getConfig('cache', 'type')))
die('Parameter [cache] => "type" is not a valid string! Please check "config.ini.php"!');
self::reportConfigurationError('cache', 'type', 'Is not a valid string');
if(!is_bool(self::getConfig('cache', 'custom_timeout')))
die('Parameter [cache] => "custom_timeout" is not a valid Boolean! Please check "config.ini.php"!');
self::reportConfigurationError('cache', 'custom_timeout', 'Is not a valid Boolean');
/** True if the cache timeout can be specified by the user */
define('CUSTOM_CACHE_TIMEOUT', self::getConfig('cache', 'custom_timeout'));
if(!is_bool(self::getConfig('authentication', 'enable')))
die('Parameter [authentication] => "enable" is not a valid Boolean! Please check "config.ini.php"!');
self::reportConfigurationError('authentication', 'enable', 'Is not a valid Boolean');
if(!is_string(self::getConfig('authentication', 'username')))
die('Parameter [authentication] => "username" is not a valid string! Please check "config.ini.php"!');
self::reportConfigurationError('authentication', 'username', 'Is not a valid string');
if(!is_string(self::getConfig('authentication', 'password')))
die('Parameter [authentication] => "password" is not a valid string! Please check "config.ini.php"!');
self::reportConfigurationError('authentication', 'password', 'Is not a valid string');
if(!empty(self::getConfig('admin', 'email'))
&& !filter_var(self::getConfig('admin', 'email'), FILTER_VALIDATE_EMAIL))
die('Parameter [admin] => "email" is not a valid email address! Please check "config.ini.php"!');
self::reportConfigurationError('admin', 'email', 'Is not a valid email address');
}
@@ -246,4 +246,46 @@ final class Configuration {
return Configuration::$VERSION;
}
/**
* Reports an configuration error for the specified section and key to the
* user and ends execution
*
* @param string $section The section name
* @param string $key The configuration key
* @param string $message An optional message to the user
*
* @return void
*/
private static function reportConfigurationError($section, $key, $message = '') {
$report = "Parameter [{$section}] => \"{$key}\" is invalid!" . PHP_EOL;
if(file_exists(FILE_CONFIG)) {
$report .= 'Please check your configuration file at ' . FILE_CONFIG . PHP_EOL;
} elseif(!file_exists(FILE_CONFIG_DEFAULT)) {
$report .= 'The default configuration file is missing at ' . FILE_CONFIG_DEFAULT . PHP_EOL;
} else {
$report .= 'The default configuration file is broken.' . PHP_EOL
. 'Restore the original file from ' . REPOSITORY . PHP_EOL;
}
$report .= $message;
self::reportError($report);
}
/**
* Reports an error message to the user and ends execution
*
* @param string $message The error message
*
* @return void
*/
private static function reportError($message) {
header('Content-Type: text/plain', true, 500);
die('Configuration error' . PHP_EOL . $message);
}
}

View File

@@ -11,6 +11,15 @@
* @link https://github.com/rss-bridge/rss-bridge
*/
/**
* Builds a GitHub search query to find open bugs for the current bridge
*/
function buildGitHubSearchQuery($bridgeName){
return REPOSITORY
. 'issues?q='
. urlencode('is:issue is:open ' . $bridgeName);
}
/**
* Returns an URL that automatically populates a new issue on GitHub based
* on the information provided
@@ -83,7 +92,8 @@ function buildBridgeException($e, $bridge){
. '`';
$body_html = nl2br($body);
$link = buildGitHubIssueQuery($title, $body, 'bug report', $bridge->getMaintainer());
$link = buildGitHubIssueQuery($title, $body, 'Bridge-Broken', $bridge->getMaintainer());
$searchQuery = buildGitHubSearchQuery($bridge::NAME);
$header = buildHeader($e, $bridge);
$message = <<<EOD
@@ -91,7 +101,7 @@ function buildBridgeException($e, $bridge){
remote website's content!<br>
{$body_html}
EOD;
$section = buildSection($e, $bridge, $message, $link);
$section = buildSection($e, $bridge, $message, $link, $searchQuery);
return $section;
}
@@ -119,11 +129,12 @@ function buildTransformException($e, $bridge){
. (isset($_SERVER['QUERY_STRING']) ? $_SERVER['QUERY_STRING'] : '')
. '`';
$link = buildGitHubIssueQuery($title, $body, 'bug report', $bridge->getMaintainer());
$link = buildGitHubIssueQuery($title, $body, 'Bridge-Broken', $bridge->getMaintainer());
$searchQuery = buildGitHubSearchQuery($bridge::NAME);
$header = buildHeader($e, $bridge);
$message = "RSS-Bridge was unable to transform the contents returned by
<strong>{$bridge->getName()}</strong>!";
$section = buildSection($e, $bridge, $message, $link);
$section = buildSection($e, $bridge, $message, $link, $searchQuery);
return buildPage($title, $header, $section);
}
@@ -154,11 +165,12 @@ EOD;
* @param object $bridge The bridge object
* @param string $message The message to display
* @param string $link The link to include in the anchor
* @param string $searchQuery A GitHub search query for the current bridge
* @return string The HTML section
*
* @todo This function belongs inside a class
*/
function buildSection($e, $bridge, $message, $link){
function buildSection($e, $bridge, $message, $link, $searchQuery){
return <<<EOD
<section>
<p class="exception-message">{$message}</p>
@@ -166,9 +178,13 @@ function buildSection($e, $bridge, $message, $link){
<ul class="advice">
<li>Press Return to check your input parameters</li>
<li>Press F5 to retry</li>
<li>Check if this issue was already reported on <a href="{$searchQuery}">GitHub</a> (give it a thumbs-up)</li>
<li>Open a <a href="{$link}">GitHub Issue</a> if this error persists</li>
</ul>
</div>
<a href="{$searchQuery}" title="Opens GitHub to search for similar issues">
<button>Search GitHub Issues</button>
</a>
<a href="{$link}" title="After clicking this button you can review
the issue before submitting it"><button>Open GitHub Issue</button></a>
<p class="maintainer">{$bridge->getMaintainer()}</p>

View File

@@ -76,7 +76,7 @@ class FeedItem {
* $item['uri'] = 'https://www.github.com/rss-bridge/rss-bridge/';
* $item['title'] = 'Title';
* $item['timestamp'] = strtotime('now');
* $item['autor'] = 'Unknown author';
* $item['author'] = 'Unknown author';
* $item['content'] = 'Hello World!';
* $item['enclosures'] = array('https://github.com/favicon.ico');
* $item['categories'] = array('php', 'rss-bridge', 'awesome');
@@ -347,7 +347,7 @@ class FeedItem {
$enclosure,
FILTER_VALIDATE_URL, FILTER_FLAG_PATH_REQUIRED)) {
Debug::log('Each enclosure must contain a scheme, host and path!');
} else {
} elseif(!in_array($enclosure, $this->enclosures)) {
$this->enclosures[] = $enclosure;
}
}
@@ -418,6 +418,9 @@ class FeedItem {
if(!is_string($uid)) {
Debug::log('Unique id must be a string!');
} elseif (preg_match('/^[a-f0-9]{40}$/', $uid)) {
// keep id if it already is a SHA-1 hash
$this->uid = $uid;
} else {
$this->uid = sha1($uid);
}

View File

@@ -195,16 +195,14 @@ class ParameterValidator {
foreach($set as $id => $properties) {
if(isset($data[$id]) && !empty($data[$id])) {
$queriedContexts[$context] = true;
} elseif(isset($properties['required'])
&& $properties['required'] === true
&& isset($properties['type'])
&& $properties['type'] !== 'checkbox'
&& $properties['type'] !== 'list') {
} elseif (isset($properties['type'])
&& ($properties['type'] === 'checkbox' || $properties['type'] === 'list')) {
continue;
} elseif(isset($properties['required']) && $properties['required'] === true) {
$queriedContexts[$context] = false;
break;
}
}
}
// Abort if one of the globally required parameters is not satisfied

View File

@@ -46,11 +46,11 @@ function getContents($url, $header = array(), $opts = array()){
// Initialize cache
$cache = Cache::create(Configuration::getConfig('cache', 'type'));
$cache->setPath(PATH_CACHE . 'server/');
$cache->setScope('server');
$cache->purgeCache(86400); // 24 hours (forced)
$params = [$url];
$cache->setParameters($params);
$cache->setKey($params);
// Use file_get_contents if in CLI mode with no root certificates defined
if(php_sapi_name() === 'cli' && empty(ini_get('curl.cainfo'))) {
@@ -207,14 +207,15 @@ EOD
* @return string Contents as simplehtmldom object.
*/
function getSimpleHTMLDOM($url,
$header = array(),
$opts = array(),
$lowercase = true,
$forceTagsClosed = true,
$target_charset = DEFAULT_TARGET_CHARSET,
$stripRN = true,
$defaultBRText = DEFAULT_BR_TEXT,
$defaultSpanText = DEFAULT_SPAN_TEXT){
$header = array(),
$opts = array(),
$lowercase = true,
$forceTagsClosed = true,
$target_charset = DEFAULT_TARGET_CHARSET,
$stripRN = true,
$defaultBRText = DEFAULT_BR_TEXT,
$defaultSpanText = DEFAULT_SPAN_TEXT){
$content = getContents($url, $header, $opts);
return str_get_html($content,
$lowercase,
@@ -256,24 +257,25 @@ $defaultSpanText = DEFAULT_SPAN_TEXT){
* @return string Contents as simplehtmldom object.
*/
function getSimpleHTMLDOMCached($url,
$duration = 86400,
$header = array(),
$opts = array(),
$lowercase = true,
$forceTagsClosed = true,
$target_charset = DEFAULT_TARGET_CHARSET,
$stripRN = true,
$defaultBRText = DEFAULT_BR_TEXT,
$defaultSpanText = DEFAULT_SPAN_TEXT){
$duration = 86400,
$header = array(),
$opts = array(),
$lowercase = true,
$forceTagsClosed = true,
$target_charset = DEFAULT_TARGET_CHARSET,
$stripRN = true,
$defaultBRText = DEFAULT_BR_TEXT,
$defaultSpanText = DEFAULT_SPAN_TEXT){
Debug::log('Caching url ' . $url . ', duration ' . $duration);
// Initialize cache
$cache = Cache::create(Configuration::getConfig('cache', 'type'));
$cache->setPath(PATH_CACHE . 'pages/');
$cache->setScope('pages');
$cache->purgeCache(86400); // 24 hours (forced)
$params = [$url];
$cache->setParameters($params);
$cache->setKey($params);
// Determine if cached file is within duration
$time = $cache->getTime();

View File

@@ -26,23 +26,13 @@
* already removes some of the tags (search for `remove_noise` in simple_html_dom.php).
*/
function sanitize($html,
$tags_to_remove = array('script', 'iframe', 'input', 'form'),
$attributes_to_keep = array('title', 'href', 'src'),
$text_to_keep = array()){
$tags_to_remove = array('script', 'iframe', 'input', 'form'),
$attributes_to_keep = array('title', 'href', 'src'),
$text_to_keep = array()){
$htmlContent = str_get_html($html);
/*
* Notice: simple_html_dom currently doesn't support "->find(*)", which is a
* known issue: https://sourceforge.net/p/simplehtmldom/bugs/157/
*
* A solution to this is to find all nodes WITHOUT a specific attribute. If
* the attribute is very unlikely to appear in the DOM, this is essentially
* returning all nodes.
*
* "*[!b38fd2b1fe7f4747d6b1c1254ccd055e]" is doing exactly that. The attrib
* "b38fd2b1fe7f4747d6b1c1254ccd055e" is very unlikely to appear in any DOM.
*/
foreach($htmlContent->find('*[!b38fd2b1fe7f4747d6b1c1254ccd055e]') as $element) {
foreach($htmlContent->find('*') as $element) {
if(in_array($element->tag, $text_to_keep)) {
$element->outertext = $element->plaintext;
} elseif(in_array($element->tag, $tags_to_remove)) {
@@ -89,18 +79,7 @@ function backgroundToImg($htmlContent) {
$regex = '/background-image[ ]{0,}:[ ]{0,}url\([\'"]{0,}(.*?)[\'"]{0,}\)/';
$htmlContent = str_get_html($htmlContent);
/*
* Notice: simple_html_dom currently doesn't support "->find(*)", which is a
* known issue: https://sourceforge.net/p/simplehtmldom/bugs/157/
*
* A solution to this is to find all nodes WITHOUT a specific attribute. If
* the attribute is very unlikely to appear in the DOM, this is essentially
* returning all nodes.
*
* "*[!b38fd2b1fe7f4747d6b1c1254ccd055e]" is doing exactly that. The attrib
* "b38fd2b1fe7f4747d6b1c1254ccd055e" is very unlikely to appear in any DOM.
*/
foreach($htmlContent->find('*[!b38fd2b1fe7f4747d6b1c1254ccd055e]') as $element) {
foreach($htmlContent->find('*') as $element) {
if(preg_match($regex, $element->style, $matches) > 0) {

View File

@@ -15,28 +15,37 @@
define('PATH_ROOT', __DIR__ . '/../');
/** Path to the core library */
define('PATH_LIB', __DIR__ . '/../lib/'); // Path to core library
define('PATH_LIB', PATH_ROOT . 'lib/');
/** Path to the vendor library */
define('PATH_LIB_VENDOR', __DIR__ . '/../vendor/');
define('PATH_LIB_VENDOR', PATH_ROOT . 'vendor/');
/** Path to the bridges library */
define('PATH_LIB_BRIDGES', __DIR__ . '/../bridges/');
define('PATH_LIB_BRIDGES', PATH_ROOT . 'bridges/');
/** Path to the formats library */
define('PATH_LIB_FORMATS', __DIR__ . '/../formats/');
define('PATH_LIB_FORMATS', PATH_ROOT . 'formats/');
/** Path to the caches library */
define('PATH_LIB_CACHES', __DIR__ . '/../caches/');
define('PATH_LIB_CACHES', PATH_ROOT . 'caches/');
/** Path to the actions library */
define('PATH_LIB_ACTIONS', __DIR__ . '/../actions/');
define('PATH_LIB_ACTIONS', PATH_ROOT . 'actions/');
/** Path to the cache folder */
define('PATH_CACHE', __DIR__ . '/../cache/');
define('PATH_CACHE', PATH_ROOT . 'cache/');
/** Path to the whitelist file */
define('WHITELIST', __DIR__ . '/../whitelist.txt');
define('WHITELIST', PATH_ROOT . 'whitelist.txt');
/** Path to the default whitelist file */
define('WHITELIST_DEFAULT', PATH_ROOT . 'whitelist.default.txt');
/** Path to the configuration file */
define('FILE_CONFIG', PATH_ROOT . 'config.ini.php');
/** Path to the default configuration file */
define('FILE_CONFIG_DEFAULT', PATH_ROOT . 'config.default.ini.php');
/** URL to the RSS-Bridge repository */
define('REPOSITORY', 'https://github.com/RSS-Bridge/rss-bridge/');

View File

@@ -84,6 +84,12 @@ input[type="number"]:focus {
border-color: #888;
}
input:focus::-webkit-input-placeholder { opacity: 0; }
input:focus::-moz-placeholder { opacity: 0; }
input:focus::placeholder { opacity: 0; }
input:focus:-moz-placeholder { opacity: 0; }
input:focus:-ms-input-placeholder { opacity: 0; }
.searchbar {
width: 40%;
margin: 40px auto 100px;
@@ -101,13 +107,6 @@ input[type="number"]:focus {
text-align: center;
}
.searchbar input[type="text"]:focus::-webkit-input-placeholder,
.searchbar input[type="text"]:focus::-moz-placeholder,
.searchbar input[type="text"]:focus:-moz-placeholder,
.searchbar input[type="text"]:focus:-ms-input-placeholder {
opacity: 0;
}
.searchbar > h3 {
font-size: 200%;
font-weight: bold;

21
vendor/simplehtmldom/LICENSE vendored Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2019 S.C. Chen, John Schlick, logmanoriginal
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

File diff suppressed because it is too large Load Diff

15
whitelist.default.txt Normal file
View File

@@ -0,0 +1,15 @@
Bandcamp
Cryptome
DansTonChat
DuckDuckGo
Facebook
Flickr
GoogleSearch
Identica
Instagram
OpenClassrooms
Pinterest
Scmb
Twitter
Wikipedia
Youtube