1
0
mirror of https://github.com/RSS-Bridge/rss-bridge.git synced 2025-08-18 06:11:33 +02:00

Compare commits

..

191 Commits

Author SHA1 Message Date
Dag
a535121ab1 Merge remote-tracking branch 'origin/master' into fix1 2022-04-08 22:37:16 +02:00
dag
cce11964a4 feat: add a timeout option for http client (#2600) 2022-04-08 21:22:13 +02:00
Corentin Garcia
8c18c02c65 [GatesNotesBridge] Add feedaxpander bridge for Bill Gate's blog (fix issue #2386) (#2611) 2022-04-08 21:21:13 +02:00
Antoine Turmel
51d27300be [FeedMergeBridge] Add new bridge (#1385)
* [FeedMergeBridge] Add new bridge

Here is a bridge that merges two or more feeds into one.

Co-authored-by: Bocki <henning@bocklage.com>
Co-authored-by: Dag <me@dvikan.no>
2022-04-08 21:13:05 +02:00
Dag
b55c5090e6 fix: require curl extension 2022-04-08 19:54:08 +02:00
quickwick
c0e2a430ab [TwitterV2Bridge] Add parameter to include only media tweets (#2614) 2022-04-07 09:00:28 +02:00
quickwick
daae089299 [TwitterV2Bridge] Changes to parameters and output titles (#2612) 2022-04-06 20:56:56 +02:00
Joseph
d98add2cac [TelegramBridge] Fix issues & add support for location messages (#2133) 2022-04-06 10:15:21 +02:00
Tobias Alexander Franke
a3b0b91dee [BinanceBridge] Remove announcements because of Cloudflare issue (#2610) 2022-04-05 23:20:27 +02:00
langfingaz
6ffe531e4f [UnsplashBridge] extend functionality (#1813) 2022-04-05 15:00:10 +02:00
Bocki
fb28107cc4 [Core] Fix prtester context issue (#2609) 2022-04-05 14:46:42 +02:00
Bocki
2c50bbae95 [AssociatedPressNewsBridge] fix checks (#2608) 2022-04-05 14:37:15 +02:00
Joseph
8f9314947b [AssociatedPressNewsBridge] Add bridge (#1475) 2022-04-05 14:03:25 +02:00
Mikalai Daronin
b24cdd47f0 [AlfaBankByBridge] new bridge for alfabank.by (#2349) 2022-04-05 13:14:09 +02:00
Tomer Shvueli
233a3cb643 general: Added a button to install RSS Bridge on Cloudron (#2559) 2022-04-05 12:22:47 +02:00
Bocki
91c6645fc7 [core] fix testing changes (#2607) 2022-04-05 12:17:19 +02:00
quickwick
780581939a [TwitterV2Bridge] New Bridge for Twitter v2 API (#2471)
* New Bridge for Twitter using v2 API

* Top comment block, tweaks to match contributing guide

* [TwitterV2Bridge] new Bridge (sort of)

* Discovered the point of, and re-added, no image scaling option

* Fix the phpcs sniff violations (I hope)

* More linter fixes, I figured out how to use phpcs locally

* Removed unnecessary custom version of getContents function

* Limit query to 100 tweets, valid example query, improved error handling

* Added config doc (correctly, I hope) with link from DESCRIPTION

* little tweak to doc
2022-04-04 21:13:05 +02:00
arnd-s
0d305f1530 [TwitterBridge] Migration to API V1.1 (#2433) 2022-04-04 19:50:59 +02:00
Nemo
e1e9a12440 [AmazonPriceTrackerBridge] Minor fix for parser, and new strategy (#2603) 2022-04-04 19:41:40 +02:00
Bocki
d34b94848b [Core] Adapt list behavior (#2605) 2022-04-04 19:40:46 +02:00
Nemo
2eaf48de99 Fix AppleAppStoreBridge (#2604) 2022-04-04 19:05:52 +02:00
LogMANOriginal
d3bb00f754 docs: Explain loadCacheValue and saveCacheValue
This adds documentation for methods added via #1380.
2022-04-03 12:19:13 +02:00
dag
00a3f80ac4 [Mangareader] chore: remove dead bridge (#2597)
It's currently timing out.
2022-04-03 10:26:17 +02:00
dag
260fc41d72 [RTFB] chore: remove dead and unmaintained bridge (#2596) 2022-04-03 10:23:06 +02:00
quickwick
28f5066fc4 Delete broken, unneeded bridges (#2595) 2022-04-03 10:10:56 +02:00
Michael Bemmerl
aa83a990d1 [OtrkeyFinderBridge] Remove HTML in title (#2594)
* [OtrkeyFinderBridge] Provide a better example that actually returns results.

* [OtrkeyFinderBridge] Remove HTML in filename.
2022-04-03 10:09:42 +02:00
Yaman Qalieh
7dcf09a876 [GitHub] Allow custom search query (#2593) 2022-04-03 10:07:35 +02:00
somini
d123e6007e Fixup deprecations on PHP 8 (#2592)
* Fixup deprecations on PHP 8

Fix #2448

* Configure a default fallback for getInput function

* Appease phpcs

* Avoid changing getInput function

Revert "Configure a default fallback for getInput function"

This reverts commit 94004c5104.
2022-04-03 09:53:13 +02:00
quickwick
a5eb02d3c3 [MixcloudBridge] switch to using API (#2591)
* switch to using public API

* switch to different API endpoints

* fix: urlencode username

Co-authored-by: Dag <me@dvikan.no>
2022-04-03 09:51:41 +02:00
dag
7b168a29f0 [WordPressPluginUpdate] fix: broken bridge (#2572)
I think they removed the changelog html page. Or maybe it
was a redirect. Anyways, this change uses their json api
to fetch plugin data.
2022-04-03 09:38:34 +02:00
dag
bed20e9f28 feat: extract curl ua to config value (#2568)
* exclude config.default.ini.php from phpcs
2022-04-03 09:37:39 +02:00
Joseph
42788cd3ee [YahtzeeDevDiaryBridge] Remove bridge (#2580)
Website has rss feeds. https://www.escapistmagazine.com/category/yahtzees-dev-diary/feed/
2022-04-02 21:11:34 +02:00
Yaman Qalieh
fb0e7ede89 [ParksOnTheAirBridge] Fix links (#2590) 2022-04-02 12:53:10 +02:00
LogMANOriginal
f311fb8083 [BridgeAbstract] Add loadCacheValue() and saveCacheValue() (#1380)
* [BridgeAbstract] Add loadCacheValue() and saveCacheValue()

Bridges currently need to implement value caching manually, which
results in duplicate code and more complex bridges.

This commit adds two protected functions to BridgeAbstract that make
it possible for bridges to store and retrieve values from a temporary
cache by key.

Co-Authored-By: Roliga <roliga.here@gmail.com>

Co-authored-by: Roliga <roliga.here@gmail.com>
2022-04-02 08:15:28 +02:00
Jacob Zelek
40a4e7b7c2 [ParksOnTheAir] New bridge for amateur radio (#2086) 2022-04-01 20:17:00 +02:00
Yaman Qalieh
73cc791ce1 [MangaDexBridge] Add new bridge (#2583) 2022-04-01 20:15:47 +02:00
Yaman Qalieh
d4707fc119 [CraigslistBridge] Fix notice with nearby results (#2588)
If the search query includes searchNearby=1, nearby results do not have
.result-hood to indicate location, instead using .nearby.
2022-04-01 16:38:37 +02:00
Yaman Qalieh
8aa091beda [GithubIssueBridge] Fix notice with reviews (#2589)
Some timeline items, like review threads and the first comment on PRs,
have no header, so this handles the first comment and adds a generic
title if that doesn't work.
2022-04-01 16:38:07 +02:00
Foxocube
d6695c0e73 [FurAffinityUserBridge] Replate username/password with cookie login (#1641) 2022-03-31 20:28:46 +02:00
Bocki
b6798b9878 general: doc fix (#2586) 2022-03-31 20:27:36 +02:00
Bocki
6baf38f251 general: fix doc (#2585) 2022-03-31 20:17:40 +02:00
Bocki
e6ae91b4d0 [FuraffinityuserBridge] Add doc about login (#2584) 2022-03-31 20:13:19 +02:00
Joseph
e525b5b427 [OpenClassroomsBridge] Remove bridge (#2582) 2022-03-31 19:41:59 +02:00
dag
983df45264 [CourrierInternationalBridge] fix: don't break on unusual feed items #2570 (#2571)
* [CourrierInternationalBridge] fix: skip unusual feed items #2570

This skips feed items who don't have content.
The one I encountered was a horoscope.
This change makes sure the bridge dont errors out.
2022-03-31 17:01:11 +02:00
dag
8717c33646 [Glassdoor] fix: repair broken bridge (#2577) 2022-03-31 17:00:14 +02:00
Joseph
7280ed7df7 [ScribdBridge] Update example profile URL value (#2578) 2022-03-31 15:58:23 +02:00
Joseph
d6b431a34b [DownDetectorBridge] Remove bridge (#2579) 2022-03-31 15:33:33 +02:00
Teemu Ikonen
aa0aa727ad [Arte7Bridge] Support all languages (#2543) 2022-03-31 11:17:07 +02:00
dag
06ef3946cd [PokemonTV] fix: use exampleValue that returns items (#2573) 2022-03-31 09:55:55 +02:00
dag
e94d447727 [DaveRamseyBlogBridge] fix: remove dead bridge #2345 (#2574) 2022-03-31 09:52:28 +02:00
dag
25e9f69261 [ElsevierBridge] fix: broken bridge (#2575) 2022-03-31 09:49:30 +02:00
dag
3e363bbc20 [FootitoBridge] chore: remove bridge (#2576) 2022-03-31 09:46:04 +02:00
Matt DeMoss
cf2dad3ab8 Reducer (retrying after failed tests) (#2273) 2022-03-30 01:50:07 +02:00
floviolleau
d6a4f2fd5b [VieDeMerdeBridge] fix due to website changes (#2567) 2022-03-30 00:58:29 +02:00
Yaman Qalieh
d27c1a99c2 [YeggiBridge] Add model source and tags (#2566) 2022-03-30 00:57:25 +02:00
Yaman Qalieh
0d80f2d5c3 [YeggiBridge] Extend description for discovery (#2565) 2022-03-29 23:48:28 +02:00
Joseph
a485beadd7 [FlickrBridge] Add content option to By username (#1861) 2022-03-29 23:46:55 +02:00
dag
ec7d2a4afb [QPlayBridge] chore: remove dead bridge (#2564) 2022-03-29 23:33:46 +02:00
dag
427becf441 [ThingiverseBridge] chore: remove dead bridge (#2563) 2022-03-29 23:13:14 +02:00
dag
267fdb27fc chore: remove dead bridge (#2562) 2022-03-29 23:12:47 +02:00
Bocki
ac242609f4 [core] Update simplehtmldom to latest released (#2556) 2022-03-29 22:45:26 +02:00
dag
461269195b fix: ignore partial json_encode() errors in JsonFormat (#2554)
Without this change, JsonFormat simply returns
an empty array. #2283
2022-03-29 22:45:00 +02:00
dag
060b4c7d58 [AnimeUltimeBridge] fix: convert strings from iso-8859-1 to utf8 (#2552)
This fixes a bug with json_encode() being unable to produce output
because it expects utf8 strings.
2022-03-29 22:44:43 +02:00
dag
cd174c7e22 [DanbooruBridge] refactor: remove unnecessary fork of simplehtmldom (#2550) 2022-03-29 22:44:20 +02:00
quickwick
907d09f116 [GelbooruBridge] + inheriting Bridges. Switch to using Gelbooru API (#2472) 2022-03-29 22:42:09 +02:00
DRogueRonin
c6675ddeee [GroupBundNaturschutzBridge] Add bridge and adjust XPathAbstract (#2445) 2022-03-29 22:40:31 +02:00
Yaman Qalieh
98a0c2de55 [EtsyBridge] Repair bridge and flip checkbox (#2457) 2022-03-29 22:23:14 +02:00
KN4CK3R
a746987d7a [Webfail] Extract timestamp from element (#1852)
Works only for German language.
2022-03-29 20:46:55 +02:00
Michael Bemmerl
6d4155f995 [GithubTrendingBridge] Fix bridge: not all languages worked (#1615)
* [GithubTrendingBridge] Fix bridge: not all languages worked

Languages with more than one word (like "Common Lisp") were not working. Looks like GitHub changed the parameter format: white space is encoded with dashes.

This prompted me to update all languages while I was at it. This also fixed the bug that the C# & F# languages were not working, because the # has to be URL encoded, which is now done in the parameter value. The language "Ren'Py" was commented out. Probably because the single quote was not escaped? I also fixed that.

* [GithubTrendingBridge] Fix PHP notice.

A repo owner can leave the repo description empty, which means the HTML element isn't there. In this case the code produced a PHP notice. This is fixed by checking for null.

* Changed getName() to retrieve the language name directly from the PARAMETERS.

Co-authored-by: dag <me@dvikan.no>
2022-03-29 20:15:18 +02:00
Bocki
58f9e41e0b [core] Change comment behavior (#2558) 2022-03-28 23:04:38 +02:00
csisoap
e86ce338a2 [ReutersBridge] Updated 'Top News' feed, some fix (#2488) 2022-03-28 20:34:41 +02:00
Mickaël Schoentgen
626cc9119a Update CryptomeBridge.php (#2555) 2022-03-28 17:18:17 +02:00
Bocki
44af64d3aa [Docker] Debug addition fixed (#2551) 2022-03-28 01:30:24 +02:00
dag
90db8c4969 [WordpressBridge] fix: add css selector for article, #2173 (#2545)
* [Wordpress] fix: add css selector for article, #2173

* fix: resolve relative links in item content
2022-03-28 00:20:44 +02:00
Bocki
8e423277e0 [core] Update pr html generator (#2549) 2022-03-27 23:35:13 +02:00
Glandos
fe43537b45 [PhoronixBridge] support multipage and embed benchmarks (#2522) 2022-03-27 13:45:32 +02:00
Thibault Couraud
87533222c7 [FindACrewBridge] Fix bridge (#2541) 2022-03-26 19:10:48 +01:00
Stelfux
91b8e4196e [FeedExpander.php] Preserve original icon (#2145) 2022-03-26 19:09:27 +01:00
Dag
74ec1b5687 [LWNprevBridge] fix: broken bridge 2022-03-26 03:17:46 +01:00
Dag
94e6feced2 Merge branch 'master' of github.com:RSS-Bridge/rss-bridge 2022-03-26 02:31:23 +01:00
Dag
b144ab2bd7 [HeiseBrige] fix: broken bridge
This is a feed expander and heise sometimes includes
feed items which point to https://www.techstage.de
for which we dont have parsing for.
2022-03-26 02:30:21 +01:00
Yaman Qalieh
012ecf8e52 [GoogleGroupsBridge] Add new bridge for Google Groups (#2451) 2022-03-26 01:09:33 +01:00
Dag
4d4ce3f380 [Arte7Bridge] test: use legal default value for checkbox 2022-03-26 00:07:34 +01:00
Dag
2c00ecb923 [Arte7Bridge] feat: add duration filter #662
The feed item was given a "duration" key but that's not used
for anything.

refs https://github.com/RSS-Bridge/rss-bridge/issues/662
2022-03-26 00:03:38 +01:00
quickwick
02ba3adcc9 [EZTVBridge] Switch to using EZTV API (#2476) 2022-03-25 22:21:47 +01:00
quickwick
37e3d6f2f6 [CBCEditorsBlogBridge] New bridge (#2487) 2022-03-25 21:35:06 +01:00
Joseph
7f4a0fae0c [YouTubeCommunityTabBridge] Add Bridge (#1594) 2022-03-25 21:31:39 +01:00
Dag
33da1476c9 [AcrimedBridge] feat: add limit option
This change preserves the prior behavior of
fetching all items.

This particular feed always has exactly 10 items.
2022-03-25 20:59:23 +01:00
Eugene Molotov
364cc8d0b8 [docs] InstagramBridge: adapt bridge documentation to new documentation structure (#2538) 2022-03-26 00:07:18 +05:00
Bocki
4c947211d2 [core] prtester debug mode (#2537) 2022-03-25 19:56:17 +01:00
Bocki
c46ff51c51 [core] Adapt pr tester 2022-03-25 19:38:26 +01:00
Bocki
608723f95c [core] Adapt pr tester (#2536) 2022-03-25 19:31:11 +01:00
Eugene Molotov
25081eedba [InstagramBridge] Documentation for configuring this bridge (#2437) 2022-03-25 23:25:35 +05:00
Bocki
aff442de1b [core] Add pr-html-generator (#2525) 2022-03-25 16:56:38 +01:00
Yaman Qalieh
105fbe9dda [PlantUMLReleasesBridge] Bridge optimizations (#2459) 2022-03-25 16:44:42 +01:00
Dag
3187592dba fix: add a few example/default values 2022-03-25 15:33:34 +01:00
Dag
2bd3f22dd5 [IPBBridge] fix: bug in feed detection logic
The previous author forgot to also append ".xml"
when actually fetching the detected feed.
2022-03-25 14:49:57 +01:00
Eugene Molotov
35b905c074 [core] Re-enable phpunit tests (#2393) 2022-03-25 12:09:05 +01:00
Yaman Qalieh
197149d90b [CraigslistBridge] Add new bridge (#2479) 2022-03-25 10:58:54 +01:00
Bocki
f11e792f84 [maintenance] Fix tests (#2532) 2022-03-25 10:41:27 +01:00
Jacob Zelek
071412130b [SummitsOnTheAir] New bridge (#2096) 2022-03-25 09:48:55 +01:00
Dag
8b59772be3 [ElsevierBridge] fix: typo in exampleValue 2022-03-25 03:42:05 +01:00
Dag
6e0589f9a0 [EconomistBridge] fix: broken bridge
Fixes: Call to a member function find() on boo

The new-style articles had their DOM changed.
2022-03-25 03:07:35 +01:00
Dag
b57d19b29c [DesoutterBridge] fix: default value 2022-03-25 02:29:51 +01:00
Dag
dbd480e2c0 [AsahiShimbunAJWBridge] fix: use case sensitive list value
This bug happened not in web UI but when extracting the default
value in order to do automatic testing.
2022-03-25 02:04:55 +01:00
Dag
35afee6103 [ABCNews] fix: broken css selector 2022-03-25 01:41:40 +01:00
Yaman Qalieh
32a6348418 [NordbayernBridge] Fix linting issue (#2531) 2022-03-25 01:26:57 +01:00
quickwick
b5ab2ee676 Add stickers endpoint for search (#2483) 2022-03-25 01:25:53 +01:00
Yaman Qalieh
acef0ab5cc [WallpaperStopBridge] Delete bridge (#2458)
This website is no longer serving content
2022-03-25 01:05:05 +01:00
Yaman Qalieh
e0d99f2a84 [DavesTrailerPageBridge] Add timestamps to feed (#2456) 2022-03-25 01:02:16 +01:00
Niehztog
55acf661b9 add support for more media types as enclosures, handle result of /tex… (#2324) 2022-03-25 00:30:14 +01:00
eggwhalefrog
3a9e528301 [NordbayernBridge] add author & timestamp of article (#2309) 2022-03-25 00:28:06 +01:00
Florent Machen
297a6cf191 [WorldCosplayBridge] fix Cosplayer API response structure (#2307) 2022-03-25 00:27:23 +01:00
Florent Machen
9cd8e93bb9 [GQMagazineBridge] fix retrieve the content of an article at a given url (#2305) 2022-03-25 00:26:38 +01:00
Dag
943a5e3e8b [CryptomeBridge] Fix pageformat (#2239) 2022-03-25 00:11:28 +01:00
Yaman Qalieh
2ade568a84 [WikipediaBridge] Add Russian Version (#2529) 2022-03-25 00:02:38 +01:00
Dag
50bab079e1 feat: add new bridge HashnodeBridge (#2231)
https://github.com/RSS-Bridge/rss-bridge/pull/2231
2022-03-24 23:58:17 +01:00
Yaman Qalieh
bb06826680 [FolhaDeSaoPauloBridge] Fix Linting Issue (#2528) 2022-03-24 23:36:23 +01:00
Dag
534864f47b Revert "WikipediaBridge: Added russian version (#2184)"
Was buggy.

This reverts commit f7af2beb79.
2022-03-24 23:34:43 +01:00
NikNikYkt
f7af2beb79 WikipediaBridge: Added russian version (#2184) 2022-03-24 23:32:33 +01:00
Yaman Qalieh
76ade41543 [No Squash] Fix Linting (#2527) 2022-03-24 23:24:55 +01:00
somini
cb4bc57c72 [FolhaDeSaoPauloBridge]: Small improvements (#1724) 2022-03-24 23:16:02 +01:00
sysadminstory
5c69577253 [ZoneTelechargementBridge] Fix links (#2526) 2022-03-24 23:13:40 +01:00
dawidsowa
78a5136cc9 bridges: change 'tags' to 'categories' (#1942) 2022-03-24 23:04:12 +01:00
Michael Bemmerl
1f2b295bf3 [BundestagParteispendenBridge] Add bridge to get the latest donations (#1613) 2022-03-24 22:42:15 +01:00
Michael Bemmerl
e89b4287b8 [SchweinfurtBuergerinformationenBridge] Add new bridge (#1610) 2022-03-24 22:37:44 +01:00
µKöff
02ab11121b [LaTeX3ProjectNewslettersBridge] New Bridge (#1589) 2022-03-24 22:26:19 +01:00
Paroleen
3d570761e5 [SpotifyBridge] Add new bridge (#1535) 2022-03-24 21:58:53 +01:00
Joseph
1ae7cf6530 [ScribdBridge] Fix bridge (#1478) 2022-03-24 21:32:16 +01:00
Jakub Valenta
8e2b65556f [Config] Don't check PATH_CACHE for memcached (#1489) 2022-03-24 21:29:16 +01:00
Joseph
0d20e9a05c [BandcampDailyBridge] Add Bridge (#1485) 2022-03-24 21:21:57 +01:00
Antoine Turmel
6a72432f76 Proposition : Open new feeds in a new tab #1389 2022-03-24 20:37:34 +01:00
Joseph
296ff9c63a [KilledbyGoogleBridge] Add bridge (#1373) 2022-03-24 20:03:18 +01:00
Bocki
2bba89d0f5 [DonnonsBridge] Fix linting error (#2524) 2022-03-24 12:15:22 +01:00
Bocki
b1c36da14e [GiphyBridge] Add examplevalue (#2523) 2022-03-24 12:11:01 +01:00
Bocki
1a8d0babd1 [Multiple] Fix all exampleValues and required variables (#2296) 2022-03-24 11:59:34 +01:00
Dag
f766193106 [IndeedBridge] fix: broken bridge
The html was reworked.
2022-03-24 13:52:02 +05:00
Dag
b6d1c7a58f fix: php notice
Fixes:
Notice: Undefined variable: message in /home/rssbridge/public/lib/contents.php on line 39
2022-03-24 13:52:02 +05:00
Dag
f34e09e93b [GithubGist] fix: broken css selector for title 2022-03-24 13:52:02 +05:00
Dag
384790537b fix: bug in cloudflare response detection
The cloudflare server header was not recognized in
some cases such as when the server header is "server"
or when the header value is "Cloudflare".
2022-03-24 13:52:02 +05:00
Dag
7bdc53125c fix: properly verify the existence of the curl module
This fixes a bug where it didnt use curl from cli
even though it's installed.

I believe this preserves the original intention to
not require the curl module to be installed.

https://github.com/RSS-Bridge/rss-bridge/pull/979
2022-03-24 13:52:02 +05:00
Binnette
076c413d3e Fix DonnonsBridge image not showing (#2521) 2022-03-24 13:51:55 +05:00
mw80
26f0380aaa [InstagramBridge] Add detectParameters (#1476) 2022-03-23 08:09:59 +05:00
Dag
14a7516625 feat: add new bridge StandfordSIRbookreviewBridge (#1638) 2022-03-23 00:43:26 +01:00
Dag
c30c0200d5 [TheTVDBBridge] fix: remove dead bridge
https://github.com/RSS-Bridge/rss-bridge/pull/1482
2022-03-23 00:07:41 +01:00
Dag
e01d9d1700 Squashed commit of the following:
commit 81d7934ab9
Author: µKöff <muekoeff@muekev.de>
Date:   Thu May 28 14:55:00 2020 +0200

    [FunkBridge] New Bridge
2022-03-22 22:42:15 +01:00
Joseph
d41aa84b13 [BridgeCard] Use full bridge name in data-ref tag (#1560)
Updates the data-ref tag of each bridge card to use the bridge's full name (eg. Apple Music) instead of its filename (eg. AppleMusic). This fixes issues with the search not returned some bridges.
2022-03-22 22:08:26 +01:00
Dag
6211a2cd37 [TagBoardBridge] fix: remove dead bridge
https://github.com/RSS-Bridge/rss-bridge/pull/1474
2022-03-22 21:38:16 +01:00
Bocki
76f5de3d0f [Documentation] Move all wiki pages into the repo and make it pretty (#2494) 2022-03-22 21:33:29 +01:00
t0stiman
16470e8119 [CarThrottleBridge] add bridge for carthrottle.com (#2514) 2022-03-22 21:15:40 +01:00
Dag
1fd3b12512 [ExecuteProgramBridge] style: remove execution bits 2022-03-22 21:11:18 +01:00
Dag
5aa163e7d6 [GettrBridge] fix: don't use php7.2 feature
The constant "JSON_THROW_ON_ERROR" is not present in
PHP version 7.2 or earlier
2022-03-22 21:08:15 +01:00
Bocki
ec90bd905e Add debug case (#2292) 2022-03-22 20:47:40 +01:00
dag
b646afffff [ExecuteProgramBridge] Add new bridge for www.executeprogram.com (#2339) 2022-03-22 20:46:59 +01:00
dag
05c31f49ce [ETTVBridge] fix: remove bridge ETTVBridge (#2511)
They went dead in Feb 2022.

Piracy Icon ETTV Officially Shuts Down Due to a Lack of Funds.

https://torrentfreak.com/piracy-icon-ettv-officially-shuts-down-due-to-a-lack-of-funds-220206/
2022-03-22 20:43:55 +01:00
dag
0b123ef8be [GettrBridge] Add new bridge for gettr.com (#2495) (#2505) 2022-03-22 20:43:31 +01:00
dag
cd5c59b84c [ARDMediathekBridge] remove timezone modification (#2507) 2022-03-22 20:42:54 +01:00
dag
e8db2479b5 [GithubTrendingBridge] fix: the description selector was broken (#2513) 2022-03-22 20:41:59 +01:00
dag
c87f4631f2 [core] feat: improve date rendering in html formatter (#2516) 2022-03-22 20:41:13 +01:00
Bockiii
ac8e94ec56 [EconomistBridge] Fix for new layout (#2489) 2022-03-23 00:24:07 +05:00
dag
1a3419a2d4 [GiphyBridge] Lazy load images (#2512)
This change instructs browsers to gradually load images
as the user is scrolling down. This is good for performance
because browsers wont download all images right away.
2022-03-21 00:43:25 +05:00
sysadminstory
ad6549efec [ZoneTelechargementBridge] (#2503)
The website keeps activating the Cloudflare protection even on the "non"
standard URL.

To bypass the Cloudflare protection, the bridge tries to resolve a bucnh
of known subdomains, and check if the IP is in a Cloudflare IP range.

If one of the subdomains has a "non Cloudflare" IP, then we use the
CURL_RESOLVE Curl option to connect the unprotected URL using the real
IP instead of the Cloudflare IP.

I hope this will help the bridge to work without needing a fix every week !
2022-03-18 14:01:46 +05:00
dag
3638b5553a phpcs: allow short array syntax (#2506)
Short array syntax was added to PHP in PHP 5.4 (2012) and replaces array() with []
2022-03-18 13:58:47 +05:00
User123698745
a7e70926f9 [Docker file] Fix wrong version string in docker images (#2497)
do not fully ignore git directory when building docker images
".git/HEAD" and ".git/refs/heads/*" are required by "getVersion()" in "lib/Configuration.php" to build the version string
2022-03-14 05:48:40 +05:00
Paul Staroch
18504f2356 [lib/contents.php] Use variable name 'retVal' instead of 'retval' as variable names are case-sensitive (#2498) 2022-03-14 05:46:30 +05:00
Eugene Molotov
05273a9278 [core] Make getContents exceptions to be handled correctly and correct exception message (#2447)
This commit fixes following issues:
1. 'Unexpected response' error message was returned, even if upstream did not return anything
2. Inability to handle non-20x messages with checking response body
2022-03-12 01:18:01 +05:00
Tomasz Kane
2e88955648 [CdactionBridge] Add missing channels (#2477) 2022-03-10 02:03:21 +05:00
Mynacol
cbef3b3360 [HeiseBridge] Properly extract authors (#2466) 2022-03-05 23:51:03 +05:00
Jonathan Kay
9564e9291f [ComicsKingdomBridge] Grab the last meta og:url content instead of first (#2484)
There are now two og:url values on the page, the first no longer contains the necessary date URL for the bridge to work.  Finding the last og:url on the page restores the bridge to working order.
2022-03-05 23:47:04 +05:00
sysadminstory
ad1ef3425a [ZoneTelechargementBridge] Fix protected links URL (#2481)
Links URL have been changed: the rewriteProtectedLink function is now updated !
2022-03-03 10:57:12 +05:00
csisoap
3bd4b0d6ab [ReutersBridge] Fix unexpected behaviour with article (#2478)
Sometimes, there are some articles that redirected to another site,
cause the bridge to fail.

Also about disable UID, Reuters frequently updated their article and my
feed reader don't update, I think it's maybe its UID.
2022-03-02 09:50:02 +05:00
Yaman Qalieh
6585ebc89b [ActionFactory] Prevent leaking working directory (#2480) 2022-03-01 14:14:53 +05:00
Corentin Garcia
d94bb08259 [RainbowSixSiegeBridge] Fix bridge (#2475) 2022-02-27 23:33:46 +05:00
Mynacol
2811bdc054 [HeiseBridge] Consistently use seite=all parameter (#2465)
This also filters out the parameter wt_mc=rss.red.ho.ho.atom.beitrag.beitrag from the item uri.
2022-02-24 23:41:42 +05:00
Joseph
0cf9da927e [CodebergBridge] Fix bridge (#2464) 2022-02-24 00:31:08 +05:00
Eugene Molotov
73a5dd928a [TwitterBridge] Don't decode HTML entities for feed content (#2470) 2022-02-24 00:28:29 +05:00
Yaman Qalieh
680fa29668 [ContainerLinuxReleasesBridge] Delete bridge (#2455)
CoreOS has been discontinued and coreos.com has been taken down (it redirects to redhat.com)
2022-02-19 17:43:04 +05:00
Loïc Fürhoff
765af484bc [RtsBridge] Add new bridge for Radio Télévision Suisse (#2442) 2022-02-17 08:15:19 +05:00
Jonathan Kay
7252252e3c [ComicsKingdomBridge] Fixes to accomodate new layout and site changes (#2444) 2022-02-13 12:27:41 +05:00
sysadminstory
3c18784576 [ZoneTelechargementBridge] Follow site changes (#2426) 2022-02-12 12:59:54 +05:00
Bockiii
3cde07db10 [Dockerfile] Rebase on php:7-apache-buster (#2446) 2022-02-12 09:27:01 +05:00
Yaman Qalieh
8723647513 [GooglePlayStoreBridge] Add bridge (#2110) 2022-02-12 09:17:12 +05:00
Eugene Molotov
f54c996e0f [CI] Add check, if php files are marked as non-executable (#2439) 2022-01-30 14:20:47 +05:00
Tomasz Kane
09fac3aa35 [CdactionBridge] Add new bridge (#2431) 2022-01-30 13:52:00 +05:00
ORelio
c1c998dd13 [GBAtempBridge] Fix content extraction (#2314)
Bridge was broken since GBAtemp's Xenforo 2 upgrade on 2021-09-23
2022-01-29 10:29:01 +05:00
Eugene Molotov
fb19142a54 [InstagramBridge] Add options to reduce 429 errors
First option is session_id of existing Instagram account.
Second option is customizing cache timeout for InstagramBridge.

Those options can be combined.
2022-01-26 00:35:15 +05:00
Eugene Molotov
9be00ff84e [core] Load bridge configuration immediately after creating bridge object
Primary reason is allowing to load configuration
params, when executing getCacheTimeout
2022-01-26 00:32:37 +05:00
Mitsu
918041cc28 [FDroid] minor syntax fix for phpcs 2022-01-24 12:41:33 +01:00
Mitsu
e9f871ce68 [FDroid] cache up, add timestamp extraction
- increase caching from 2 to 4 hours
- using cURL, extract Last-Modified header of app icons and use as item timestamp
Test warning: F-Droid response time is quite slow even on static assets, the additional requests might impact bridge performance further
2022-01-24 12:36:49 +01:00
Sandro
018fd1c8f2 [GithubPullRequestBridge] Sort by newest PRs instead of latest updated (#2064) 2022-01-24 10:58:54 +05:00
Eugene Molotov
30553d8665 contrib: lint fetch_contributors.php 2022-01-20 10:22:10 +05:00
268 changed files with 10082 additions and 3269 deletions

View File

@@ -1,4 +1,6 @@
.git
!.git/HEAD
!.git/refs/heads/*
.gitattributes
.github/*
.travis.yml

2
.github/prtester-requirements.txt vendored Normal file
View File

@@ -0,0 +1,2 @@
beautifulsoup4>=4.10.0
requests>=2.26.0

101
.github/prtester.py vendored Normal file
View File

@@ -0,0 +1,101 @@
import requests
from bs4 import BeautifulSoup
from datetime import datetime
import os.path
# This script is specifically written to be used in automation for https://github.com/RSS-Bridge/rss-bridge
#
# This will scrape the whitelisted bridges in the current state (port 3000) and the PR state (port 3001) of
# RSS-Bridge, generate a feed for each of the bridges and save the output as html files.
# It also replaces the default static CSS link with a hardcoded link to @em92's public instance, so viewing
# the HTML file locally will actually work as designed.
def testBridges(bridges,status):
for bridge in bridges:
if bridge.get('data-ref'): # Some div entries are empty, this ignores those
bridgeid = bridge.get('id')
bridgeid = bridgeid.split('-')[1] # this extracts a readable bridge name from the bridge metadata
bridgestring = '/?action=display&bridge=' + bridgeid + '&format=Html'
forms = bridge.find_all("form")
formid = 1
for form in forms:
# a bridge can have multiple contexts, named 'forms' in html
# this code will produce a fully working formstring that should create a working feed when called
# this will create an example feed for every single context, to test them all
formstring = ''
errormessages = []
parameters = form.find_all("input")
lists = form.find_all("select")
# this for/if mess cycles through all available input parameters, checks if it required, then pulls
# the default or examplevalue and then combines it all together into the formstring
# if an example or default value is missing for a required attribute, it will throw an error
# any non-required fields are not tested!!!
for parameter in parameters:
if parameter.get('type') == 'hidden' and parameter.get('name') == 'context':
cleanvalue = parameter.get('value').replace(" ","+")
formstring = formstring + '&' + parameter.get('name') + '=' + cleanvalue
if parameter.get('type') == 'number' or parameter.get('type') == 'text':
if parameter.has_attr('required'):
if parameter.get('placeholder') == '':
if parameter.get('value') == '':
errormessages.append(parameter.get('name'))
else:
formstring = formstring + '&' + parameter.get('name') + '=' + parameter.get('value')
else:
formstring = formstring + '&' + parameter.get('name') + '=' + parameter.get('placeholder')
# same thing, just for checkboxes. If a checkbox is checked per default, it gets added to the formstring
if parameter.get('type') == 'checkbox':
if parameter.has_attr('checked'):
formstring = formstring + '&' + parameter.get('name') + '=on'
for list in lists:
selectionvalue = ''
for selectionentry in list.contents:
if 'selected' in selectionentry.attrs:
selectionvalue = selectionentry.get('value')
break
if selectionvalue == '':
selectionvalue = list.contents[0].get('value')
formstring = formstring + '&' + list.get('name') + '=' + selectionvalue
if not errormessages:
# if all example/default values are present, form the full request string, run the request, replace the static css
# file with the url of em's public instance and then upload it to termpad.com, a pastebin-like-site.
r = requests.get(URL + bridgestring + formstring)
pagetext = r.text.replace('static/HtmlFormat.css','https://feed.eugenemolotov.ru/static/HtmlFormat.css')
pagetext = pagetext.encode("utf_8")
termpad = requests.post(url="https://termpad.com/", data=pagetext)
termpadurl = termpad.text
termpadurl = termpadurl.replace('termpad.com/','termpad.com/raw/')
termpadurl = termpadurl.replace('\n','')
with open(os.getcwd() + '/comment.txt', 'a+') as file:
file.write("\n")
file.write("| [`" + bridgeid + '-' + status + '-context' + str(formid) + "`](" + termpadurl + ") | " + date_time + " |")
else:
# if there are errors (which means that a required value has no example or default value), log out which error appeared
termpad = requests.post(url="https://termpad.com/", data=str(errormessages))
termpadurl = termpad.text
termpadurl = termpadurl.replace('termpad.com/','termpad.com/raw/')
termpadurl = termpadurl.replace('\n','')
with open(os.getcwd() + '/comment.txt', 'a+') as file:
file.write("\n")
file.write("| [`" + bridgeid + '-' + status + '-context' + str(formid) + "`](" + termpadurl + ") | " + date_time + " |")
formid += 1
gitstatus = ["current", "pr"]
now = datetime.now()
date_time = now.strftime("%Y-%m-%d, %H:%M:%S")
with open(os.getcwd() + '/comment.txt', 'w+') as file:
file.write(''' ## Pull request artifacts
| file | last change |
| ---- | ------ |''')
for status in gitstatus: # run this twice, once for the current version, once for the PR version
if status == "current":
port = "3000" # both ports are defined in the corresponding workflow .yml file
elif status == "pr":
port = "3001"
URL = "http://localhost:" + port
page = requests.get(URL) # Use python requests to grab the rss-bridge main page
soup = BeautifulSoup(page.content, "html.parser") # use bs4 to turn the page into soup
bridges = soup.find_all("section") # get a soup-formatted list of all bridges on the rss-bridge page
testBridges(bridges,status) # run the main scraping code with the list of bridges and the info if this is for the current version or the pr version

27
.github/workflows/documentation.yml vendored Normal file
View File

@@ -0,0 +1,27 @@
name: Documentation
on:
push:
paths:
- 'docs/**'
jobs:
documentation:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
persist-credentials: false
- name: Setup PHP
uses: shivammathur/setup-php@2.17.1
with:
php-version: 8.0
- name: Install dependencies
run: composer global require daux/daux.io
- name: Generate documentation
run: daux generate
- name: Deploy same repository 🚀
uses: JamesIves/github-pages-deploy-action@v4.2.5
with:
folder: "static"
branch: gh-pages

View File

@@ -33,3 +33,16 @@ jobs:
- run: composer global require dealerdirect/phpcodesniffer-composer-installer
- run: composer global require phpcompatibility/php-compatibility
- run: ~/.composer/vendor/bin/phpcs . --standard=phpcompatibility.xml --warning-severity=0 --extensions=php -p
executable_php_files_check:
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2
- run: |
if find -name "*.php" -executable -type f -print -exec false {} +
then
echo 'Good, no executable php scripts found'
else
echo 'Please unmark php scripts above as non-executable'
exit 1
fi

69
.github/workflows/prhtmlgenerator.yml vendored Normal file
View File

@@ -0,0 +1,69 @@
name: 'PR Testing'
on:
pull_request_target:
branches: [ master ]
jobs:
test-pr:
name: Generate HTML
runs-on: ubuntu-latest
# Needs additional permissions https://github.com/actions/first-interaction/issues/10#issuecomment-1041402989
steps:
- name: Check out self
uses: actions/checkout@v2.3.2
with:
ref: ${{github.event.pull_request.head.ref}}
repository: ${{github.event.pull_request.head.repo.full_name}}
- name: Check out rss-bridge
run: |
PR=${{github.event.number}};
wget -O requirements.txt https://raw.githubusercontent.com/RSS-Bridge/rss-bridge/master/.github/prtester-requirements.txt;
wget https://raw.githubusercontent.com/RSS-Bridge/rss-bridge/master/.github/prtester.py;
wget https://patch-diff.githubusercontent.com/raw/$GITHUB_REPOSITORY/pull/$PR.patch;
touch DEBUG;
cat $PR.patch | grep " bridges/.*\.php" | sed "s= bridges/\(.*\)Bridge.php.*=\1=g" | sort | uniq > whitelist.txt
- name: Start Docker - Current
run: |
docker run -d -v $GITHUB_WORKSPACE/whitelist.txt:/app/whitelist.txt -v $GITHUB_WORKSPACE/DEBUG:/app/DEBUG -p 3000:80 ghcr.io/rss-bridge/rss-bridge:latest
- name: Start Docker - PR
run: |
docker build -t prbuild .;
docker run -d -v $GITHUB_WORKSPACE/whitelist.txt:/app/whitelist.txt -v $GITHUB_WORKSPACE/DEBUG:/app/DEBUG -p 3001:80 prbuild
- name: Setup python
uses: actions/setup-python@v2
with:
python-version: '3.7'
cache: 'pip'
- name: Install requirements
run: |
cd $GITHUB_WORKSPACE
pip install -r requirements.txt
- name: Run bridge tests
id: testrun
run: |
mkdir results;
python prtester.py;
body="$(cat comment.txt)";
body="${body//'%'/'%25'}";
body="${body//$'\n'/'%0A'}";
body="${body//$'\r'/'%0D'}";
echo "::set-output name=bodylength::${#body}"
echo "::set-output name=body::$body"
- name: Find Comment
if: ${{ steps.testrun.outputs.bodylength > 130 }}
uses: peter-evans/find-comment@v2
id: fc
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: 'github-actions[bot]'
body-includes: Pull request artifacts
- name: Create or update comment
if: ${{ steps.testrun.outputs.bodylength > 130 }}
uses: peter-evans/create-or-update-comment@v2
with:
comment-id: ${{ steps.fc.outputs.comment-id }}
issue-number: ${{ github.event.pull_request.number }}
body: |
${{ steps.testrun.outputs.body }}
edit-mode: replace

View File

@@ -7,19 +7,18 @@ on:
branches: [ master ]
jobs:
# TODO: return back when fixed https://github.com/RSS-Bridge/rss-bridge/issues/2391
# phpunit7:
# runs-on: ubuntu-20.04
# strategy:
# matrix:
# php-versions: ['7.1', '7.2', '7.3']
# steps:
# - uses: actions/checkout@v2
# - uses: shivammathur/setup-php@v2
# with:
# php-version: ${{ matrix.php-versions }}
# - run: composer global require phpunit/phpunit ^7
# - run: phpunit --configuration=phpunit.xml --include-path=lib/
phpunit7:
runs-on: ubuntu-20.04
strategy:
matrix:
php-versions: ['7.1', '7.2', '7.3']
steps:
- uses: actions/checkout@v2
- uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php-versions }}
- run: composer global require phpunit/phpunit ^7
- run: phpunit --configuration=phpunit.xml --include-path=lib/
phpunit8:
runs-on: ubuntu-20.04

1
.gitignore vendored
View File

@@ -213,6 +213,7 @@ pip-log.txt
# Unit test / coverage reports
.coverage
.phpunit.result.cache
.tox
#Translations

View File

@@ -1,4 +1,4 @@
FROM php:7-apache
FROM php:7-apache-buster
LABEL description="RSS-Bridge is a PHP project capable of generating RSS and Atom feeds for websites that don't have one."
LABEL repository="https://github.com/RSS-Bridge/rss-bridge"

View File

@@ -22,7 +22,7 @@ Supported sites/pages (examples)
* `FlickrExplore` : [Latest interesting images](http://www.flickr.com/explore) from Flickr
* `GoogleSearch` : Most recent results from Google Search
* `Identi.ca` : Identica user timeline (Should be compatible with other Pump.io instances)
* `Instagram`: Most recent photos from an Instagram user (There is an [issue](https://github.com/RSS-Bridge/rss-bridge/issues/1891) for public instances)
* `Instagram`: Most recent photos from an Instagram user (It is recommended to [configure](https://rss-bridge.github.io/rss-bridge/Bridge_Specific/Instagram.html) this bridge to work)
* `OpenClassrooms`: Lastest tutorials from [fr.openclassrooms.com](http://fr.openclassrooms.com/)
* `Pinterest`: Most recent photos from user or search
* `ScmbBridge`: Newest stories from [secouchermoinsbete.fr](http://secouchermoinsbete.fr/)
@@ -88,9 +88,11 @@ Deploy
===
Thanks to the community, hosting your own instance of RSS-Bridge is as easy as clicking a button!
*Note: External providers' applications are packaged by 3rd parties. Use at your own discretion.*
[![Deploy on Scalingo](https://cdn.scalingo.com/deploy/button.svg)](https://my.scalingo.com/deploy?source=https://github.com/sebsauvage/rss-bridge)
[![Deploy to Heroku](https://www.herokucdn.com/deploy/button.svg)](https://heroku.com/deploy)
[![Deploy to Cloudron](https://cloudron.io/img/button.svg)](https://www.cloudron.io/store/com.rssbridgeapp.cloudronapp.html)
Getting involved
===
@@ -189,6 +191,7 @@ Use this script to generate the list automatically (using the GitHub API):
* [hunhejj](https://github.com/hunhejj)
* [husim0](https://github.com/husim0)
* [IceWreck](https://github.com/IceWreck)
* [imagoiq](https://github.com/imagoiq)
* [j0k3r](https://github.com/j0k3r)
* [JackNUMBER](https://github.com/JackNUMBER)
* [jacquesh](https://github.com/jacquesh)
@@ -223,7 +226,7 @@ Use this script to generate the list automatically (using the GitHub API):
* [m0zes](https://github.com/m0zes)
* [Mar-Koeh](https://github.com/Mar-Koeh)
* [marcus-at-localhost](https://github.com/marcus-at-localhost)
* [marius851000](https://github.com/marius851000)
* [marius8510000-bot](https://github.com/marius8510000-bot)
* [matthewseal](https://github.com/matthewseal)
* [mcbyte-it](https://github.com/mcbyte-it)
* [mdemoss](https://github.com/mdemoss)
@@ -238,6 +241,7 @@ Use this script to generate the list automatically (using the GitHub API):
* [mro](https://github.com/mro)
* [mschwld](https://github.com/mschwld)
* [mxmehl](https://github.com/mxmehl)
* [Mynacol](https://github.com/Mynacol)
* [nel50n](https://github.com/nel50n)
* [niawag](https://github.com/niawag)
* [Niehztog](https://github.com/Niehztog)
@@ -294,6 +298,8 @@ Use this script to generate the list automatically (using the GitHub API):
* [thezeroalpha](https://github.com/thezeroalpha)
* [timendum](https://github.com/timendum)
* [TitiTestScalingo](https://github.com/TitiTestScalingo)
* [tomaszkane](https://github.com/tomaszkane)
* [TReKiE](https://github.com/TReKiE)
* [triatic](https://github.com/triatic)
* [VerifiedJoseph](https://github.com/VerifiedJoseph)
* [WalterBarrett](https://github.com/WalterBarrett)

View File

@@ -38,6 +38,7 @@ class DisplayAction extends ActionAbstract {
// Data retrieval
$bridge = $bridgeFac->create($bridge);
$bridge->loadConfiguration();
$noproxy = array_key_exists('_noproxy', $this->userData)
&& filter_var($this->userData['_noproxy'], FILTER_VALIDATE_BOOLEAN);
@@ -131,7 +132,6 @@ class DisplayAction extends ActionAbstract {
try {
$bridge->setDatas($bridge_params);
$bridge->loadConfiguration();
$bridge->collectData();
$items = $bridge->getItems();

View File

@@ -27,7 +27,7 @@ class ABCNewsBridge extends BridgeAbstract {
public function collectData() {
$url = 'https://www.abc.net.au/news/' . $this->getInput('topic');
$html = getSimpleHTMLDOM($url)->find('.YAJzu._26IxR._2kxNB._3BZxh', 0);
$html = getSimpleHTMLDOM($url)->find('.YAJzu._2FvRw.ZWhbj._3BZxh', 0);
$html = defaultLinkTo($html, $this->getURI());
foreach($html->find('._2H7Su') as $article) {

View File

@@ -1,42 +0,0 @@
<?php
class ABCTabsBridge extends BridgeAbstract {
const MAINTAINER = 'kranack';
const NAME = 'ABC Tabs Bridge';
const URI = 'https://www.abc-tabs.com/';
const DESCRIPTION = 'Returns 22 newest tabs';
public function collectData(){
$html = '';
$html = getSimpleHTMLDOM(static::URI . 'tablatures/nouveautes.html')
or returnClientError('No results for this query.');
$table = $html->find('table#myTable', 0)->children(1);
foreach ($table->find('tr') as $tab) {
$item = array();
$item['author'] = $tab->find('td', 1)->plaintext
. ' - '
. $tab->find('td', 2)->plaintext;
$item['title'] = $tab->find('td', 1)->plaintext
. ' - '
. $tab->find('td', 2)->plaintext;
$item['content'] = 'Le '
. $tab->find('td', 0)->plaintext
. '<br> Par: '
. $tab->find('td', 5)->plaintext
. '<br> Type: '
. $tab->find('td', 3)->plaintext;
$item['id'] = static::URI
. $tab->find('td', 2)->find('a', 0)->getAttribute('href');
$item['uri'] = static::URI
. $tab->find('td', 2)->find('a', 0)->getAttribute('href');
$this->items[] = $item;
}
}
}

View File

@@ -12,8 +12,7 @@ class AO3Bridge extends BridgeAbstract {
'name' => 'url',
'required' => true,
// Example: F/F tag, complete works only
'exampleValue' => self::URI
. 'works?work_search[complete]=T&tag_id=F*s*F',
'exampleValue' => 'https://archiveofourown.org/works?work_search[complete]=T&tag_id=F*s*F',
),
),
'Bookmarks' => array(

View File

@@ -51,6 +51,8 @@ class ARDMediathekBridge extends BridgeAbstract {
);
public function collectData() {
$oldTz = date_default_timezone_get();
date_default_timezone_set('Europe/Berlin');
$pathComponents = explode('/', $this->getInput('path'));
@@ -87,5 +89,7 @@ class ARDMediathekBridge extends BridgeAbstract {
$item['author'] = $video->publicationService->name;
$this->items[] = $item;
}
date_default_timezone_set($oldTz);
}
}

View File

@@ -3,12 +3,25 @@ class AcrimedBridge extends FeedExpander {
const MAINTAINER = 'qwertygc';
const NAME = 'Acrimed Bridge';
const URI = 'http://www.acrimed.org/';
const URI = 'https://www.acrimed.org/';
const CACHE_TIMEOUT = 4800; //2hours
const DESCRIPTION = 'Returns the newest articles';
const PARAMETERS = [
[
'limit' => [
'name' => 'limit',
'type' => 'number',
'defaultValue' => -1,
]
]
];
public function collectData(){
$this->collectExpandableDatas(static::URI . 'spip.php?page=backend');
$this->collectExpandableDatas(
static::URI . 'spip.php?page=backend',
$this->getInput('limit')
);
}
protected function parseItem($newsItem){

View File

@@ -11,6 +11,7 @@ class AlbionOnlineBridge extends BridgeAbstract {
'postcount' => array(
'name' => 'Limit',
'type' => 'number',
'required' => true,
'title' => 'Maximum number of items to return',
'defaultValue' => 5,
),

View File

@@ -0,0 +1,83 @@
<?php
class AlfaBankByBridge extends BridgeAbstract {
const MAINTAINER = 'lassana';
const NAME = 'AlfaBank.by Новости';
const URI = 'https://www.alfabank.by';
const DESCRIPTION = 'Уведомления Alfa-Now — новости от Альфа-Банка';
const CACHE_TIMEOUT = 3600; // 1 hour
const PARAMETERS = array(
'News' => array(
'business' => array(
'name' => 'Альфа Бизнес',
'type' => 'list',
'title' => 'В зависимости от выбора, возращает уведомления для" .
" клиентов физ. лиц либо для клиентов-юридических лиц и ИП',
'values' => array(
'Новости' => 'news',
'Новости бизнеса' => 'newsBusiness'
),
'defaultValue' => 'news'
),
'fullContent' => array(
'name' => 'Включать содержимое',
'type' => 'checkbox',
'title' => 'Если выбрано, содержимое уведомлений вставляется в поток (работает медленно)'
)
)
);
public function collectData() {
$business = $this->getInput('business') == 'newsBusiness';
$fullContent = $this->getInput('fullContent') == 'on';
$mainPageUrl = self::URI . '/about/articles/uvedomleniya/';
if($business) {
$mainPageUrl .= '?business=true';
}
$html = getSimpleHTMLDOM($mainPageUrl);
$limit = 0;
foreach($html->find('a.notifications__item') as $element) {
if($limit < 10) {
$item = array();
$item['uid'] = 'urn:sha1:' . hash('sha1', $element->getAttribute('data-notification-id'));
$item['title'] = $element->find('div.item-title', 0)->innertext;
$item['timestamp'] = DateTime::createFromFormat(
'd M Y',
$this->ruMonthsToEn($element->find('div.item-date', 0)->innertext)
)->getTimestamp();
$itemUrl = self::URI . $element->href;
if($business) {
$itemUrl = str_replace('?business=true', '', $itemUrl);
}
$item['uri'] = $itemUrl;
if($fullContent) {
$itemHtml = getSimpleHTMLDOM($itemUrl);
if($itemHtml) {
$item['content'] = $itemHtml->find('div.now-p__content-text', 0)->innertext;
}
}
$this->items[] = $item;
$limit++;
}
}
}
public function getIcon() {
return static::URI . '/local/images/favicon.ico';
}
private function ruMonthsToEn($date) {
$ruMonths = array(
'Января', 'Февраля', 'Марта', 'Апреля', 'Мая', 'Июня',
'Июля', 'Августа', 'Сентября', 'Октября', 'Ноября', 'Декабря' );
$enMonths = array(
'January', 'February', 'March', 'April', 'May', 'June',
'July', 'August', 'September', 'October', 'November', 'December' );
return str_replace($ruMonths, $enMonths, $date);
}
}

View File

@@ -4,7 +4,7 @@ class AllocineFRBridge extends BridgeAbstract {
const MAINTAINER = 'superbaillot.net';
const NAME = 'Allo Cine Bridge';
const CACHE_TIMEOUT = 25200; // 7h
const URI = 'http://www.allocine.fr/';
const URI = 'https://www.allocine.fr/';
const DESCRIPTION = 'Bridge for allocine.fr';
const PARAMETERS = array( array(
'category' => array(

View File

@@ -12,6 +12,7 @@ class AmazonBridge extends BridgeAbstract {
'q' => array(
'name' => 'Keyword',
'required' => true,
'exampleValue' => 'watch',
),
'sort' => array(
'name' => 'Sort by',

View File

@@ -49,6 +49,8 @@ class AmazonPriceTrackerBridge extends BridgeAbstract {
'.a-color-price',
);
const WHITESPACE = " \t\n\r\0\x0B\xC2\xA0";
protected $title;
/**
@@ -154,6 +156,22 @@ EOT;
return false;
}
private function scrapePriceTwister($html) {
$str = $html->find('.twister-plus-buying-options-price-data', 0);
$data = json_decode($str->innertext, true);
if(count($data) === 1) {
$data = $data[0];
return array(
'displayPrice' => $data['displayPrice'],
'currency' => $data['currency'],
'shipping' => '0',
);
}
return false;
}
private function scrapePriceGeneric($html) {
$priceDiv = null;
@@ -168,12 +186,11 @@ EOT;
return false;
}
$priceString = $priceDiv->plaintext;
preg_match('/[\d.,]+/', $priceString, $matches);
$priceString = str_replace(str_split(self::WHITESPACE), '', $priceDiv->plaintext);
preg_match('/(\d+\.\d{0,2})/', $priceString, $matches);
$price = $matches[0];
$currency = trim(str_replace($price, '', $priceString), " \t\n\r\0\x0B\xC2\xA0");
$currency = str_replace($price, '', $priceString);
if ($price != null && $currency != null) {
return array(
@@ -186,6 +203,21 @@ EOT;
return false;
}
private function renderContent($image, $data) {
$price = $data['displayPrice'];
if (!$price) {
$price = "{$data['price']} {$data['currency']}";
}
$html = "$image<br>Price: $price";
if ($data['shipping'] !== '0') {
$html .= "<br>Shipping: {$data['shipping']} {$data['currency']}</br>";
}
return $html;
}
/**
* Scrape method for Amazon product page
* @return [type] [description]
@@ -195,20 +227,16 @@ EOT;
$this->title = $this->getTitle($html);
$imageTag = $this->getImage($html);
$data = $this->scrapePriceFromMetrics($html) ?: $this->scrapePriceGeneric($html);
$data = $this->scrapePriceGeneric($html);
$item = array(
'title' => $this->title,
'uri' => $this->getURI(),
'content' => "$imageTag<br/>Price: {$data['price']} {$data['currency']}",
'content' => $this->renderContent($imageTag, $data),
// This is to ensure that feed readers notice the price change
'uid' => md5($data['price'])
);
if ($data['shipping'] !== '0') {
$item['content'] .= "<br>Shipping: {$data['shipping']} {$data['currency']}</br>";
}
$this->items[] = $item;
}
}

View File

@@ -37,9 +37,11 @@ class AnimeUltimeBridge extends BridgeAbstract {
$processedOK = 0;
foreach (array($thismonth, $lastmonth) as $requestFilter) {
//Retrive page contents
$url = self::URI . 'history-0-1/' . $requestFilter;
$html = getSimpleHTMLDOM($url);
$html = getContents($url);
// Convert html from iso-8859-1 => utf8
$html = utf8_encode($html);
$html = str_get_html($html);
//Relases are sorted by day : process each day individually
foreach($html->find('div.history', 0)->find('h3') as $daySection) {
@@ -87,6 +89,8 @@ class AnimeUltimeBridge extends BridgeAbstract {
// Retrieve description from description page
$html_item = getContents($item_uri);
// Convert html from iso-8859-1 => utf8
$html_item = utf8_encode($html_item);
$item_description = substr(
$html_item,
strpos($html_item, 'class="principal_contain" align="center">') + 41

View File

@@ -94,6 +94,7 @@ class AppleAppStoreBridge extends BridgeAbstract {
$headers = array(
"Authorization: Bearer $token",
'Origin: https://apps.apple.com',
);
$json = json_decode(getContents($uri, $headers), true);

View File

@@ -10,7 +10,8 @@ class ArtStationBridge extends BridgeAbstract {
'Search Query' => array(
'q' => array(
'name' => 'Search term',
'required' => true
'required' => true,
'exampleValue' => 'bird'
)
)
);

View File

@@ -10,79 +10,82 @@ class Arte7Bridge extends BridgeAbstract {
const API_TOKEN = 'Nzc1Yjc1ZjJkYjk1NWFhN2I2MWEwMmRlMzAzNjI5NmU3NWU3ODg4ODJjOWMxNTMxYzEzZGRjYjg2ZGE4MmIwOA';
const PARAMETERS = array(
'Catégorie (Français)' => array(
'catfr' => array(
'global' => [
'video_duration_filter' => [
'name' => 'Exclude short videos',
'type' => 'checkbox',
'title' => 'Exclude videos that are shorter than 3 minutes',
'defaultValue' => false,
],
],
'Category' => array(
'lang' => array(
'type' => 'list',
'name' => 'Catégorie',
'name' => 'Language',
'values' => array(
'Toutes les vidéos (français)' => null,
'Actu & société' => 'ACT',
'Séries & fiction' => 'SER',
'Cinéma' => 'CIN',
'Arts & spectacles classiques' => 'ARS',
'Français' => 'fr',
'Deutsch' => 'de',
'English' => 'en',
'Español' => 'es',
'Polski' => 'pl',
'Italiano' => 'it'
),
'title' => 'ex. RC-014095 pour https://www.arte.tv/fr/videos/RC-014095/blow-up/',
'exampleValue' => 'RC-014095'
),
'cat' => array(
'type' => 'list',
'name' => 'Category',
'values' => array(
'All videos' => null,
'News & society' => 'ACT',
'Series & fiction' => 'SER',
'Cinema' => 'CIN',
'Culture' => 'ARS',
'Culture pop' => 'CPO',
'Découverte' => 'DEC',
'Histoire' => 'HIST',
'Discovery' => 'DEC',
'History' => 'HIST',
'Science' => 'SCI',
'Autre' => 'AUT'
'Other' => 'AUT'
)
)
),
),
'Collection (Français)' => array(
'colfr' => array(
'name' => 'Collection id',
'required' => true,
'title' => 'ex. RC-014095 pour https://www.arte.tv/fr/videos/RC-014095/blow-up/'
)
),
'Catégorie (Allemand)' => array(
'catde' => array(
'Collection' => array(
'lang' => array(
'type' => 'list',
'name' => 'Catégorie',
'name' => 'Language',
'values' => array(
'Alle Videos (deutsch)' => null,
'Aktuelles & Gesellschaft' => 'ACT',
'Fernsehfilme & Serien' => 'SER',
'Kino' => 'CIN',
'Kunst & Kultur' => 'ARS',
'Popkultur & Alternativ' => 'CPO',
'Entdeckung' => 'DEC',
'Geschichte' => 'HIST',
'Wissenschaft' => 'SCI',
'Sonstiges' => 'AUT'
'Français' => 'fr',
'Deutsch' => 'de',
'English' => 'en',
'Español' => 'es',
'Polski' => 'pl',
'Italiano' => 'it'
)
)
),
'Collection (Allemand)' => array(
'colde' => array(
),
'col' => array(
'name' => 'Collection id',
'required' => true,
'title' => 'ex. RC-014095 pour https://www.arte.tv/de/videos/RC-014095/blow-up/'
'title' => 'ex. RC-014095 pour https://www.arte.tv/de/videos/RC-014095/blow-up/',
'exampleValue' => 'RC-014095'
)
)
);
public function collectData(){
$lang = $this->getInput('lang');
switch($this->queriedContext) {
case 'Catégorie (Français)':
$category = $this->getInput('catfr');
$lang = 'fr';
case 'Category':
$category = $this->getInput('cat');
$collectionId = null;
break;
case 'Collection (Français)':
$lang = 'fr';
$collectionId = $this->getInput('colfr');
break;
case 'Catégorie (Allemand)':
$category = $this->getInput('catde');
$lang = 'de';
break;
case 'Collection (Allemand)':
$lang = 'de';
$collectionId = $this->getInput('colde');
case 'Collection':
$collectionId = $this->getInput('col');
$category = null;
break;
}
$url = 'https://api.arte.tv/api/opa/v3/videos?sort=-lastModified&limit=10&language='
$url = 'https://api.arte.tv/api/opa/v3/videos?sort=-lastModified&limit=15&language='
. $lang
. ($category != null ? '&category.code=' . $category : '')
. ($collectionId != null ? '&collections.collectionId=' . $collectionId : '');
@@ -95,6 +98,11 @@ class Arte7Bridge extends BridgeAbstract {
$input_json = json_decode($input, true);
foreach($input_json['videos'] as $element) {
$durationSeconds = $element['durationSeconds'];
if ($this->getInput('video_duration_filter') && $durationSeconds < 60 * 3) {
continue;
}
$item = array();
$item['uri'] = $element['url'];
@@ -106,10 +114,10 @@ class Arte7Bridge extends BridgeAbstract {
if(!empty($element['subtitle']))
$item['title'] = $element['title'] . ' | ' . $element['subtitle'];
$item['duration'] = round((int)$element['durationSeconds'] / 60);
$durationMinutes = round((int)$durationSeconds / 60);
$item['content'] = $element['teaserText']
. '<br><br>'
. $item['duration']
. $durationMinutes
. 'min<br><a href="'
. $item['uri']
. '"><img src="'

View File

@@ -26,7 +26,7 @@ class AsahiShimbunAJWBridge extends BridgeAbstract {
'Opinion » Editorial' => 'opinion/editorial',
'Opinion » Vox Populi' => 'opinion/vox',
),
'defaultValue' => 'Politics',
'defaultValue' => 'politics',
)
)
);

View File

@@ -10,7 +10,8 @@ class AskfmBridge extends BridgeAbstract {
'Ask.fm username' => array(
'u' => array(
'name' => 'Username',
'required' => true
'required' => true,
'exampleValue' => 'ApprovedAndReal'
)
)
);

View File

@@ -0,0 +1,270 @@
<?php
class AssociatedPressNewsBridge extends BridgeAbstract {
const NAME = 'Associated Press News Bridge';
const URI = 'https://apnews.com/';
const DESCRIPTION = 'Returns newest articles by topic';
const MAINTAINER = 'VerifiedJoseph';
const PARAMETERS = array(
'Standard Topics' => array(
'topic' => array(
'name' => 'Topic',
'type' => 'list',
'values' => array(
'AP Top News' => 'apf-topnews',
'Sports' => 'apf-sports',
'Entertainment' => 'apf-entertainment',
'Oddities' => 'apf-oddities',
'Travel' => 'apf-Travel',
'Technology' => 'apf-technology',
'Lifestyle' => 'apf-lifestyle',
'Business' => 'apf-business',
'U.S. News' => 'apf-usnews',
'Health' => 'apf-Health',
'Science' => 'apf-science',
'World News' => 'apf-WorldNews',
'Politics' => 'apf-politics',
'Religion' => 'apf-religion',
'Photo Galleries' => 'PhotoGalleries',
'Fact Checks' => 'APFactCheck',
'Videos' => 'apf-videos',
),
'defaultValue' => 'apf-topnews',
),
),
'Custom Topic' => array(
'topic' => array(
'name' => 'Topic',
'type' => 'text',
'required' => true,
'exampleValue' => 'europe'
),
)
);
const CACHE_TIMEOUT = 900; // 15 mins
private $detectParamRegex = '/^https?:\/\/(?:www\.)?apnews\.com\/(?:[tag|hub]+\/)?([\w-]+)$/';
private $tagEndpoint = 'https://afs-prod.appspot.com/api/v2/feed/tag?tags=';
private $feedName = '';
public function detectParameters($url) {
$params = array();
if(preg_match($this->detectParamRegex, $url, $matches) > 0) {
$params['topic'] = $matches[1];
$params['context'] = 'Custom Topic';
return $params;
}
return null;
}
public function collectData() {
switch($this->getInput('topic')) {
case 'Podcasts':
returnClientError('Podcasts topic feed is not supported');
break;
case 'PressReleases':
returnClientError('PressReleases topic feed is not supported');
break;
default:
$this->collectCardData();
}
}
public function getURI() {
if (!is_null($this->getInput('topic'))) {
return self::URI . $this->getInput('topic');
}
return parent::getURI();
}
public function getName() {
if (!empty($this->feedName)) {
return $this->feedName . ' - Associated Press';
}
return parent::getName();
}
private function getTagURI() {
if (!is_null($this->getInput('topic'))) {
return $this->tagEndpoint . $this->getInput('topic');
}
return parent::getURI();
}
private function collectCardData() {
$json = getContents($this->getTagURI())
or returnServerError('Could not request: ' . $this->getTagURI());
$tagContents = json_decode($json, true);
if (empty($tagContents['tagObjs'])) {
returnClientError('Topic not found: ' . $this->getInput('topic'));
}
$this->feedName = $tagContents['tagObjs'][0]['name'];
foreach ($tagContents['cards'] as $card) {
$item = array();
// skip hub peeks & Notifications
if ($card['cardType'] == 'Hub Peek' || $card['cardType'] == 'Notification') {
continue;
}
$storyContent = $card['contents'][0];
switch($storyContent['contentType']) {
case 'web': // Skip link only content
continue 2;
case 'video':
$html = $this->processVideo($storyContent);
$item['enclosures'][] = 'https://storage.googleapis.com/afs-prod/media/'
. $storyContent['media'][0]['id'] . '/800.jpeg';
break;
default:
if (empty($storyContent['storyHTML'])) { // Skip if no storyHTML
continue 2;
}
$html = defaultLinkTo($storyContent['storyHTML'], self::URI);
$html = str_get_html($html);
$this->processMediaPlaceholders($html, $storyContent['id']);
$this->processHubLinks($html, $storyContent);
$this->processIframes($html);
if (!is_null($storyContent['leadPhotoId'])) {
$item['enclosures'][] = 'https://storage.googleapis.com/afs-prod/media/'
. $storyContent['leadPhotoId'] . '/800.jpeg';
}
}
$item['title'] = $card['contents'][0]['headline'];
$item['uri'] = self::URI . $card['shortId'];
if ($card['contents'][0]['localLinkUrl']) {
$item['uri'] = $card['contents'][0]['localLinkUrl'];
}
$item['timestamp'] = $storyContent['published'];
if (is_null($storyContent['bylines']) === false) {
// Remove 'By' from the bylines
if (substr($storyContent['bylines'], 0, 2) == 'By') {
$item['author'] = ltrim($storyContent['bylines'], 'By ');
} else {
$item['author'] = $storyContent['bylines'];
}
}
$item['content'] = $html;
foreach ($storyContent['tagObjs'] as $tag) {
$item['categories'][] = $tag['name'];
}
$this->items[] = $item;
if (count($this->items) >= 15) {
break;
}
}
}
private function processMediaPlaceholders($html, $id) {
if ($html->find('div.media-placeholder', 0)) {
// Fetch page content
$json = getContents('https://afs-prod.appspot.com/api/v2/content/' . $id);
$storyContent = json_decode($json, true);
foreach ($html->find('div.media-placeholder') as $div) {
$key = array_search($div->id, $storyContent['mediumIds']);
if (!isset($storyContent['media'][$key])) {
continue;
}
$media = $storyContent['media'][$key];
if ($media['type'] === 'Photo') {
$mediaUrl = $media['gcsBaseUrl'] . $media['imageRenderedSizes'][0] . $media['imageFileExtension'];
$mediaCaption = $media['caption'];
$div->outertext = <<<EOD
<figure><img loading="lazy" src="{$mediaUrl}"/><figcaption>{$mediaCaption}</figcaption></figure>
EOD;
}
if ($media['type'] === 'YouTube') {
$div->outertext = <<<EOD
<iframe src="https://www.youtube.com/embed/{$media['externalId']}" width="560" height="315">
</iframe>
EOD;
}
}
}
}
/*
Create full coverage links (HubLinks)
*/
private function processHubLinks($html, $storyContent) {
if (!empty($storyContent['richEmbeds'])) {
foreach ($storyContent['richEmbeds'] as $embed) {
if ($embed['type'] === 'Hub Link') {
$url = self::URI . $embed['tag']['id'];
$div = $html->find('div[id=' . $embed['id'] . ']', 0);
if ($div) {
$div->outertext = <<<EOD
<p><a href="{$url}">{$embed['calloutText']} {$embed['displayName']}</a></p>
EOD;
}
}
}
}
}
private function processVideo($storyContent) {
$video = $storyContent['media'][0];
if ($video['type'] === 'YouTube') {
$url = 'https://www.youtube.com/embed/' . $video['externalId'];
$html = <<<EOD
<iframe width="560" height="315" src="{$url}" frameborder="0" allowfullscreen></iframe>
EOD;
} else {
$html = <<<EOD
<video controls poster="https://storage.googleapis.com/afs-prod/media/{$video['id']}/800.jpeg" preload="none">
<source src="{$video['gcsBaseUrl']} {$video['videoRenderedSizes'][0]} {$video['videoFileExtension']}" type="video/mp4">
</video>
EOD;
}
return $html;
}
// Remove datawrapper.dwcdn.net iframes and related javaScript
private function processIframes($html) {
foreach ($html->find('iframe') as $index => $iframe) {
if (preg_match('/datawrapper\.dwcdn\.net/', $iframe->src)) {
$iframe->outertext = '';
if ($html->find('script', $index)) {
$html->find('script', $index)->outertext = '';
}
}
}
}
}

View File

@@ -8,7 +8,8 @@ class AtmoOccitanieBridge extends BridgeAbstract {
const PARAMETERS = array(array(
'city' => array(
'name' => 'Ville',
'required' => true
'required' => true,
'exampleValue' => 'cahors'
)
));
const CACHE_TIMEOUT = 7200;

View File

@@ -10,7 +10,7 @@ class BakaUpdatesMangaReleasesBridge extends BridgeAbstract {
'name' => 'Series ID',
'type' => 'number',
'required' => true,
'exampleValue' => '12345'
'exampleValue' => '188066'
)
),
'By list' => array(
@@ -18,7 +18,7 @@ class BakaUpdatesMangaReleasesBridge extends BridgeAbstract {
'name' => 'List ID and Type',
'type' => 'text',
'required' => true,
'exampleValue' => '123456&list=read'
'exampleValue' => '4395&list=read'
)
)
);

View File

@@ -11,7 +11,8 @@ class BandcampBridge extends BridgeAbstract {
'tag' => array(
'name' => 'tag',
'type' => 'text',
'required' => true
'required' => true,
'exampleValue' => 'hip-hop-rap'
)
),
'By band' => array(
@@ -19,7 +20,8 @@ class BandcampBridge extends BridgeAbstract {
'name' => 'band',
'type' => 'text',
'title' => 'Band name as seen in the band page URL',
'required' => true
'required' => true,
'exampleValue' => 'aesoprock'
),
'type' => array(
'name' => 'Articles are',
@@ -34,6 +36,7 @@ class BandcampBridge extends BridgeAbstract {
'limit' => array(
'name' => 'limit',
'type' => 'number',
'required' => true,
'title' => 'Number of releases to return',
'defaultValue' => 5
)
@@ -67,13 +70,15 @@ class BandcampBridge extends BridgeAbstract {
'name' => 'band',
'type' => 'text',
'title' => 'Band name as seen in the album page URL',
'required' => true
'required' => true,
'exampleValue' => 'aesoprock'
),
'album' => array(
'name' => 'album',
'type' => 'text',
'title' => 'Album name as seen in the album page URL',
'required' => true
'required' => true,
'exampleValue' => 'appleseed'
),
'type' => array(
'name' => 'Articles are',

View File

@@ -0,0 +1,155 @@
<?php
class BandcampDailyBridge extends BridgeAbstract {
const NAME = 'Bandcamp Daily Bridge';
const URI = 'https://daily.bandcamp.com';
const DESCRIPTION = 'Returns newest articles';
const MAINTAINER = 'VerifiedJoseph';
const PARAMETERS = array(
'Latest articles' => array(),
'Best of' => array(
'content' => array(
'name' => 'content',
'type' => 'list',
'values' => array(
'Best Ambient' => 'best-ambient',
'Best Beat Tapes' => 'best-beat-tapes',
'Best Dance 12\'s' => 'best-dance-12s',
'Best Contemporary Classical' => 'best-contemporary-classical',
'Best Electronic' => 'best-electronic',
'Best Experimental' => 'best-experimental',
'Best Hip-Hop' => 'best-hip-hop',
'Best Jazz' => 'best-jazz',
'Best Metal' => 'best-metal',
'Best Punk' => 'best-punk',
'Best Reissues' => 'best-reissues',
'Best Soul' => 'best-soul',
),
'defaultValue' => 'best-ambient',
),
),
'Genres' => array(
'content' => array(
'name' => 'content',
'type' => 'list',
'values' => array(
'Acoustic' => 'genres/acoustic',
'Alternative' => 'genres/alternative',
'Ambient' => 'genres/ambient',
'Blues' => 'genres/blues',
'Classical' => 'genres/classical',
'Comedy' => 'genres/comedy',
'Country' => 'genres/country',
'Devotional' => 'genres/devotional',
'Electronic' => 'genres/electronic',
'Experimental' => 'genres/experimental',
'Folk' => 'genres/folk',
'Funk' => 'genres/funk',
'Hip-Hop/Rap' => 'genres/hip-hop-rap',
'Jazz' => 'genres/jazz',
'Kids' => 'genres/kids',
'Latin' => 'genres/latin',
'Metal' => 'genres/metal',
'Pop' => 'genres/pop',
'Punk' => 'genres/punk',
'R&B/Soul' => 'genres/r-b-soul',
'Reggae' => 'genres/reggae',
'Rock' => 'genres/rock',
'Soundtrack' => 'genres/soundtrack',
'Spoken Word' => 'genres/spoken-word',
'World' => 'genres/world',
),
'defaultValue' => 'genres/acoustic',
),
),
'Franchises' => array(
'content' => array(
'name' => 'content',
'type' => 'list',
'values' => array(
'Lists' => 'lists',
'Features' => 'features',
'Album of the Day' => 'album-of-the-day',
'Acid Test' => 'acid-test',
'Bandcamp Navigator' => 'bandcamp-navigator',
'Big Ups' => 'big-ups',
'Certified' => 'certified',
'Gallery' => 'gallery',
'Hidden Gems' => 'hidden-gems',
'High Scores' => 'high-scores',
'Label Profile' => 'label-profile',
'Lifetime Achievement' => 'lifetime-achievement',
'Scene Report' => 'scene-report',
'Seven Essential Releases' => 'seven-essential-releases',
'The Merch Table' => 'the-merch-table',
),
'defaultValue' => 'lists',
),
)
);
const CACHE_TIMEOUT = 3600; // 1 hour
public function collectData() {
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request: ' . $this->getURI());
$html = defaultLinkTo($html, self::URI);
$articles = $html->find('articles-list', 0);
foreach($articles->find('div.list-article') as $index => $article) {
$item = array();
$articlePath = $article->find('a.title', 0)->href;
$articlePageHtml = getSimpleHTMLDOMCached($articlePath, 3600)
or returnServerError('Could not request: ' . $articlePath);
$item['uri'] = $articlePath;
$item['title'] = $articlePageHtml->find('article-title', 0)->innertext;
$item['author'] = $articlePageHtml->find('article-credits > a', 0)->innertext;
$item['content'] = html_entity_decode($articlePageHtml->find('meta[name="description"]', 0)->content, ENT_QUOTES);
$item['timestamp'] = $articlePageHtml->find('meta[property="article:published_time"]', 0)->content;
$item['categories'][] = $articlePageHtml->find('meta[property="article:section"]', 0)->content;
if ($articlePageHtml->find('meta[property="article:tag"]', 0)) {
$item['categories'][] = $articlePageHtml->find('meta[property="article:tag"]', 0)->content;
}
$item['enclosures'][] = $articlePageHtml->find('meta[name="twitter:image"]', 0)->content;
$this->items[] = $item;
if (count($this->items) >= 10) {
break;
}
}
}
public function getURI() {
switch($this->queriedContext) {
case 'Latest articles':
return self::URI . '/latest';
case 'Best of':
case 'Genres':
case 'Franchises':
return self::URI . '/' . $this->getInput('content');
default:
return parent::getURI();
}
}
public function getName() {
if ($this->queriedContext === 'Latest articles') {
return $this->queriedContext . ' - Bandcamp Daily';
}
if (!is_null($this->getInput('content'))) {
$contentValues = array_flip(self::PARAMETERS[$this->queriedContext]['content']['values']);
return $contentValues[$this->getInput('content')] . ' - Bandcamp Daily';
}
return parent::getName();
}
}

View File

@@ -1,41 +1,18 @@
<?php
class BinanceBridge extends BridgeAbstract {
const NAME = 'Binance';
const URI = 'https://www.binance.com';
const DESCRIPTION = 'Subscribe to the Binance blog or the Binance Zendesk announcements.';
const NAME = 'Binance Blog';
const URI = 'https://www.binance.com/en/blog';
const DESCRIPTION = 'Subscribe to the Binance blog.';
const MAINTAINER = 'thefranke';
const CACHE_TIMEOUT = 3600; // 1h
const PARAMETERS = array( array(
'category' => array(
'name' => 'category',
'type' => 'list',
'exampleValue' => 'Blog',
'title' => 'Select a category',
'values' => array(
'Blog' => 'Blog',
'Announcements' => 'Announcements'
)
)
));
public function getIcon() {
return 'https://bin.bnbstatic.com/static/images/common/favicon.ico';
}
public function getName() {
return self::NAME . ' ' . $this->getInput('category');
}
public function getURI() {
if ($this->getInput('category') == 'Blog')
return self::URI . '/en/blog';
else
return 'https://binance.zendesk.com/hc/en-us/categories/115000056351-Announcements';
}
protected function collectBlogData() {
$html = getSimpleHTMLDOM($this->getURI());
public function collectData() {
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not fetch Binance blog data.');
$appData = $html->find('script[id="__APP_DATA"]');
$appDataJson = json_decode($appData[0]->innertext);
@@ -61,37 +38,4 @@ class BinanceBridge extends BridgeAbstract {
break;
}
}
protected function collectAnnouncementData() {
$html = getSimpleHTMLDOM($this->getURI());
foreach($html->find('a.article-list-link') as $a) {
$title = $a->innertext;
$uri = 'https://binance.zendesk.com' . $a->href;
$full = getSimpleHTMLDOMCached($uri);
$content = $full->find('div.article-body', 0);
$date = $full->find('time', 0)->getAttribute('datetime');
$item = array();
$item['title'] = $title;
$item['uri'] = $uri;
$item['timestamp'] = strtotime($date);
$item['author'] = 'Binance';
$item['content'] = $content;
$this->items[] = $item;
if (count($this->items) >= 10)
break;
}
}
public function collectData() {
if ($this->getInput('category') == 'Blog')
$this->collectBlogData();
else
$this->collectAnnouncementData();
}
}

View File

@@ -1,35 +1,62 @@
<?php
require_once('GelbooruBridge.php');
require_once('DanbooruBridge.php');
class BooruprojectBridge extends GelbooruBridge {
class BooruprojectBridge extends DanbooruBridge {
const MAINTAINER = 'mitsukarenai';
const NAME = 'Booruproject';
const URI = 'http://booru.org/';
const URI = 'https://booru.org/';
const DESCRIPTION = 'Returns images from given page of booruproject';
const PARAMETERS = array(
'global' => array(
'p' => array(
'name' => 'page',
'defaultValue' => 0,
'type' => 'number'
),
't' => array(
'name' => 'tags'
'name' => 'tags',
'required' => true,
'exampleValue' => 'tagme',
'title' => 'Use "all" to get all posts'
)
),
'Booru subdomain (subdomain.booru.org)' => array(
'i' => array(
'name' => 'Subdomain',
'required' => true
'required' => true,
'exampleValue' => 'rm'
)
)
);
const PATHTODATA = '.thumb';
const IDATTRIBUTE = 'id';
const TAGATTRIBUTE = 'title';
const PIDBYPAGE = 20;
protected function getFullURI(){
return $this->getURI()
. 'index.php?page=post&s=list&pid='
. ($this->getInput('p') ? ($this->getInput('p') - 1) * static::PIDBYPAGE : '')
. '&tags=' . urlencode($this->getInput('t'));
}
protected function getTags($element){
$tags = parent::getTags($element);
$tags = explode(' ', $tags);
// Remove statistics from the tags list (identified by colon)
foreach($tags as $key => $tag) {
if(strpos($tag, ':') !== false) unset($tags[$key]);
}
return implode(' ', $tags);
}
public function getURI(){
if(!is_null($this->getInput('i'))) {
return 'http://' . $this->getInput('i') . '.booru.org/';
return 'https://' . $this->getInput('i') . '.booru.org/';
}
return parent::getURI();

0
bridges/BukowskisBridge.php Executable file → Normal file
View File

View File

@@ -0,0 +1,89 @@
<?php
class BundestagParteispendenBridge extends BridgeAbstract {
const MAINTAINER = 'mibe';
const NAME = 'Deutscher Bundestag - Parteispenden';
const URI = 'https://www.bundestag.de/parlament/praesidium/parteienfinanzierung/fundstellen50000';
const CACHE_TIMEOUT = 86400; // 24h
const DESCRIPTION = 'Returns the latest "soft money" donations to parties represented in the German Bundestag.';
const CONTENT_TEMPLATE = <<<TMPL
<p><b>Partei:</b><br>%s</p>
<p><b>Spendenbetrag:</b><br>%s</p>
<p><b>Spender:</b><br>%s</p>
<p><b>Eingang der Spende:</b><br>%s</p>
TMPL;
public function getIcon()
{
return 'https://www.bundestag.de/static/appdata/includes/images/layout/favicon.ico';
}
public function collectData()
{
$ajaxUri = <<<URI
https://www.bundestag.de/ajax/filterlist/de/parlament/praesidium/parteienfinanzierung/fundstellen50000/462002-462002
URI;
// Get the main page
$html = getSimpleHTMLDOMCached($ajaxUri, self::CACHE_TIMEOUT)
or returnServerError('Could not request AJAX list.');
// Build the URL from the first anchor element. The list is sorted by year, descending, so the first element is the current year.
$firstAnchor = $html->find('a', 0)
or returnServerError('Could not find the proper HTML element.');
$url = 'https://www.bundestag.de' . $firstAnchor->href;
// Get the actual page with the soft money donations
$html = getSimpleHTMLDOMCached($url, self::CACHE_TIMEOUT)
or returnServerError('Could not request ' . $url);
$rows = $html->find('table.table > tbody > tr')
or returnServerError('Could not find the proper HTML elements.');
foreach($rows as $row) {
$item = $this->generateItemFromRow($row);
if (is_array($item)) {
$item['uri'] = $url;
$this->items[] = $item;
}
}
}
private function generateItemFromRow(simple_html_dom_node $row)
{
// The row must have 5 columns. There are monthly header rows, which are ignored here.
if(count($row->children) != 5)
return null;
$item = array();
// | column | paragraph inside column
$party = $row->children[0]->children[0]->innertext;
$amount = $row->children[1]->children[0]->innertext . ' €';
$donor = $row->children[2]->children[0]->innertext;
$date = $row->children[3]->children[0]->innertext;
$dip = $row->children[4]->children[0]->find('a.dipLink', 0);
// Strip whitespace from date string.
$date = str_replace(' ', '', $date);
$content = sprintf(self::CONTENT_TEMPLATE, $party, $amount, $donor, $date);
$item = array(
'title' => $party . ': ' . $amount,
'content' => $content,
'uid' => sha1($content),
);
// Try to get the link to the official document
if ($dip != null)
$item['enclosures'] = array($dip->href);
// Try to parse the date
$dateTime = DateTime::createFromFormat('d.m.Y', $date);
if ($dateTime !== false)
$item['timestamp'] = $dateTime->getTimestamp();
return $item;
}
}

View File

@@ -0,0 +1,36 @@
<?php
class CBCEditorsBlogBridge extends BridgeAbstract {
const MAINTAINER = 'quickwick';
const NAME = 'CBC Editors Blog';
const URI = 'https://www.cbc.ca/news/editorsblog';
const DESCRIPTION = 'Recent CBC Editor\'s Blog posts';
public function collectData(){
$html = getSimpleHTMLDOM(self::URI);
// Loop on each blog post entry
foreach($html->find('div.contentListCards', 0)->find('a[data-test=type-story]') as $element) {
$headline = ($element->find('.headline', 0))->innertext;
$timestamp = ($element->find('time', 0))->datetime;
$articleUri = 'https://www.cbc.ca' . $element->href;
$summary = ($element->find('div.description', 0))->innertext;
$thumbnailUris = ($element->find('img[loading=lazy]', 0))->srcset;
$thumbnailUri = rtrim(explode(',', $thumbnailUris)[0], ' 300w');
// Fill item
$item = array();
$item['uri'] = $articleUri;
$item['id'] = $item['uri'];
$item['timestamp'] = $timestamp;
$item['title'] = $headline;
$item['content'] = '<img src="'
. $thumbnailUri . '" /><br>' . $summary;
$item['author'] = 'Editor\'s Blog';
if(isset($item['title'])) {
$this->items[] = $item;
}
}
}
}

View File

@@ -12,13 +12,13 @@ class CNETFranceBridge extends FeedExpander
'name' => 'Exclude by title',
'required' => false,
'title' => 'Title term, separated by semicolon (;)',
'defaultValue' => 'bon plan;bons plans;au meilleur prix;des meilleures offres;Amazon Prime Day;RED by SFR ou B&You'
'exampleValue' => 'bon plan;bons plans;au meilleur prix;des meilleures offres;Amazon Prime Day;RED by SFR ou B&You'
),
'url' => array(
'name' => 'Exclude by url',
'required' => false,
'title' => 'URL term, separated by semicolon (;)',
'defaultValue' => 'bon-plan;bons-plans'
'exampleValue' => 'bon-plan;bons-plans'
)
)
);

View File

@@ -0,0 +1,41 @@
<?php
class CarThrottleBridge extends FeedExpander {
const NAME = 'Car Throttle ';
const URI = 'https://www.carthrottle.com';
const DESCRIPTION = 'Get the latest car-related news from Car Throttle.';
const MAINTAINER = 't0stiman';
public function collectData() {
$this->collectExpandableDatas('https://www.carthrottle.com/rss', 10);
}
protected function parseItem($feedItem) {
$item = parent::parseItem($feedItem);
//fetch page
$articlePage = getSimpleHTMLDOMCached($feedItem->link)
or returnServerError('Could not retrieve ' . $feedItem->link);
$subtitle = $articlePage->find('p.standfirst', 0);
$article = $articlePage->find('div.content_field', 0);
$item['content'] = str_get_html($subtitle . $article);
//convert <iframe>s to <a>s. meant for embedded videos.
foreach($item['content']->find('iframe') as $found) {
$iframeUrl = $found->getAttribute('src');
if ($iframeUrl) {
$found->outertext = '<a href="' . $iframeUrl . '">' . $iframeUrl . '</a>';
}
}
//remove scripts from the text
foreach ($item['content']->find('script') as $remove) {
$remove->outertext = '';
}
return $item;
}
}

View File

@@ -13,8 +13,8 @@ class CastorusBridge extends BridgeAbstract {
'name' => 'ZIP code',
'type' => 'text',
'required' => true,
'exampleValue' => '74910, 74',
'title' => 'Insert ZIP code (complete or partial)'
'exampleValue' => '7',
'title' => 'Insert ZIP code (complete or partial). e.g: 78125 OR 781 OR 7'
)
),
'Get latest changes via city name' => array(
@@ -22,8 +22,8 @@ class CastorusBridge extends BridgeAbstract {
'name' => 'City name',
'type' => 'text',
'required' => true,
'exampleValue' => 'Seyssel, Seys',
'title' => 'Insert city name (complete or partial)'
'exampleValue' => 'Paris',
'title' => 'Insert city name (complete or partial). e.g: Paris OR Par OR P'
)
)
);

View File

@@ -0,0 +1,60 @@
<?php
class CdactionBridge extends BridgeAbstract {
const NAME = 'CD-ACTION bridge';
const URI = 'https://cdaction.pl';
const DESCRIPTION = 'Fetches the latest posts from given category.';
const MAINTAINER = 'tomaszkane';
const PARAMETERS = array( array(
'category' => array(
'name' => 'Kategoria',
'type' => 'list',
'values' => array(
'Najnowsze (wszystkie)' => 'najnowsze',
'Newsy' => 'newsy',
'Recenzje' => 'recenzje',
'Teksty' => array(
'Publicystyka' => 'publicystyka',
'Zapowiedzi' => 'zapowiedzi',
'Już graliśmy' => 'juz-gralismy',
'Poradniki' => 'poradniki',
),
'Kultura' => 'kultura',
'Wideo' => 'wideo',
'Czasopismo' => 'czasopismo',
'Technologie' => array(
'Artykuły' => 'artykuly',
'Testy' => 'testy',
),
'Na luzie' => array(
'Konkursy' => 'konkursy',
'Nadgodziny' => 'nadgodziny',
)
)
))
);
public function collectData() {
$html = getSimpleHTMLDOM($this->getURI() . '/' . $this->getInput('category'));
$newsJson = $html->find('script#__NEXT_DATA__', 0)->innertext;
if (!$newsJson = json_decode($newsJson)) {
return;
}
$queriesIndex = $this->getInput('category') === 'najnowsze' ? 0 : 1;
foreach ($newsJson->props->pageProps->dehydratedState->queries[$queriesIndex]->state->data->results as $news) {
$item = array();
$item['uri'] = $this->getURI() . '/' . $news->category->slug . '/' . $news->slug;
$item['title'] = $news->title;
$item['timestamp'] = $news->publishedAt;
$item['author'] = $news->editor->fullName;
$item['content'] = $news->lead;
$item['enclosures'][] = $news->bannerUrl;
$item['categories'] = array_column($news->tags, 'name');
$item['uid'] = $news->id;
$this->items[] = $item;
}
}
}

View File

@@ -20,6 +20,7 @@ class CodebergBridge extends BridgeAbstract {
'name' => 'Issue ID',
'type' => 'text',
'required' => true,
'exampleValue' => '513',
)
),
'Pull Requests' => array(),
@@ -28,14 +29,14 @@ class CodebergBridge extends BridgeAbstract {
'username' => array(
'name' => 'Username',
'type' => 'text',
'exampleValue' => 'username',
'exampleValue' => 'Codeberg',
'title' => 'Username of account that the repository belongs to.',
'required' => true,
),
'repo' => array(
'name' => 'Repository',
'type' => 'text',
'exampleValue' => 'repo',
'exampleValue' => 'Community',
'required' => true,
)
)
@@ -205,6 +206,9 @@ class CodebergBridge extends BridgeAbstract {
return $this->getInput('username') . '/' . $this->getInput('repo');
}
/**
* Extract commits
*/
private function extractCommits($html) {
$table = $html->find('table#commits-table', 0);
$tbody = $table->find('tbody.commit-list', 0);
@@ -231,25 +235,27 @@ class CodebergBridge extends BridgeAbstract {
}
}
/**
* Extract issues
*/
private function extractIssues($html) {
$div = $html->find('div.repository', 0);
$div = $html->find('div.issue.list', 0);
foreach ($div->find('li.item') as $li) {
$item = array();
$number = $li->find('div', 0)->plaintext;
$number = trim($li->find('a.index,ml-0.mr-2', 0)->plaintext);
$item['title'] = $li->find('a.title', 0)->plaintext . ' (' . $number . ')';
$item['uri'] = $li->find('a.title', 0)->href;
$item['timestamp'] = $li->find('p.desc', 0)->find('span', 0)->title;
$item['author'] = $li->find('p.desc', 0)->find('a', 0)->plaintext;
$item['timestamp'] = $li->find('span.time-since', 0)->title;
$item['author'] = $li->find('div.desc', 0)->find('a', 1)->plaintext;
// Fetch issue page
$issuePage = getSimpleHTMLDOMCached($item['uri'], 3600);
$issuePage = defaultLinkTo($issuePage, self::URI);
$item['content'] = $issuePage->find('ui.timeline', 0)->find('div.render-content.markdown', 0);
$item['content'] = $issuePage->find('div.timeline-item.comment.first', 0)->find('div.render-content.markup', 0);
foreach ($li->find('a.ui.label') as $label) {
$item['categories'][] = $label->plaintext;
@@ -259,20 +265,23 @@ class CodebergBridge extends BridgeAbstract {
}
}
/**
* Extract issue comments
*/
private function extractIssueComments($html) {
$this->issueTitle = $html->find('span#issue-title', 0)->plaintext
. ' (' . $html->find('span.index', 0)->plaintext . ')';
foreach ($html->find('ui.timeline > div.timeline-item.comment') as $div) {
foreach ($html->find('div.timeline-item.comment') as $div) {
$item = array();
if ($div->class === 'timeline-item comment merge box') {
continue;
}
$item['title'] = $this->ellipsisTitle($div->find('div.render-content.markdown', 0)->plaintext);
$item['title'] = $this->ellipsisTitle($div->find('div.render-content.markup', 0)->plaintext);
$item['uri'] = $div->find('span.text.grey', 0)->find('a', 1)->href;
$item['content'] = $div->find('div.render-content.markdown', 0);
$item['content'] = $div->find('div.render-content.markup', 0);
if ($div->find('div.dropzone-attachments', 0)) {
$item['content'] .= $div->find('div.dropzone-attachments', 0);
@@ -285,25 +294,27 @@ class CodebergBridge extends BridgeAbstract {
}
}
/**
* Extract pulls
*/
private function extractPulls($html) {
$div = $html->find('div.repository', 0);
$div = $html->find('div.issue.list', 0);
foreach ($div->find('li.item') as $li) {
$item = array();
$number = $li->find('div', 0)->plaintext;
$number = trim($li->find('a.index,ml-0.mr-2', 0)->plaintext);
$item['title'] = $li->find('a.title', 0)->plaintext . ' (' . $number . ')';
$item['uri'] = $li->find('a.title', 0)->href;
$item['timestamp'] = $li->find('p.desc', 0)->find('span', 0)->title;
$item['author'] = $li->find('p.desc', 0)->find('a', 0)->plaintext;
$item['timestamp'] = $li->find('span.time-since', 0)->title;
$item['author'] = $li->find('div.desc', 0)->find('a', 1)->plaintext;
// Fetch pull request page
$pullRequestPage = getSimpleHTMLDOMCached($item['uri'], 3600);
$pullRequestPage = defaultLinkTo($pullRequestPage, self::URI);
$item['content'] = $pullRequestPage->find('ui.timeline', 0)->find('div.render-content.markdown', 0);
$item['content'] = $pullRequestPage->find('ui.timeline', 0)->find('div.render-content.markup', 0);
foreach ($li->find('a.ui.label') as $label) {
$item['categories'][] = $label->plaintext;
@@ -313,49 +324,40 @@ class CodebergBridge extends BridgeAbstract {
}
}
/**
* Extract releases
*/
private function extractReleases($html) {
$ul = $html->find('ul#release-list', 0);
foreach ($ul->find('li.ui.grid') as $li) {
$item = array();
$item['title'] = $li->find('h4', 0)->plaintext;
$item['uri'] = $li->find('h4', 0)->find('a', 0)->href;
if ($li->find('h3', 0)) { // Release
$item['title'] = $li->find('h3', 0)->plaintext;
$item['uri'] = $li->find('h3', 0)->find('a', 0)->href;
$tag = $this->stripSvg($li->find('span.tag', 0));
$commit = $this->stripSvg($li->find('span.commit', 0));
$downloads = $this->extractDownloads($li->find('details.download', 0));
$tag = $li->find('span.tag', 0)->find('a', 0);
$commit = $li->find('span.commit', 0);
$downloads = $this->extractDownloads($li->find('div.download', 0));
$item['content'] = $li->find('div.markdown', 0);
$item['content'] .= <<<HTML
$item['content'] = $li->find('div.markup.desc', 0);
$item['content'] .= <<<HTML
<strong>Tag</strong>
<p>{$tag}</p>
<strong>Commit</strong>
<p>{$commit}</p>
{$downloads}
HTML;
$item['timestamp'] = $li->find('span.time', 0)->find('span', 0)->title;
$item['author'] = $li->find('span.author', 0)->find('a', 0)->plaintext;
}
if ($li->find('h4', 0)) { // Tag
$item['title'] = $li->find('h4', 0)->plaintext;
$item['uri'] = $li->find('h4', 0)->find('a', 0)->href;
$item['content'] = <<<HTML
<strong>Commit</strong>
<p>{$li->find('div.download', 0)->find('a', 0)}</p>
HTML;
$item['content'] .= $this->extractDownloads($li->find('div.download', 0), true);
$item['timestamp'] = $li->find('span.time', 0)->find('span', 0)->title;
}
$item['timestamp'] = $li->find('span.time', 0)->find('span', 0)->title;
$item['author'] = $li->find('span.author', 0)->find('a', 0)->plaintext;
$this->items[] = $item;
}
}
/**
* Extract downloads for a releases
*/
private function extractDownloads($html, $skipFirst = false) {
$downloads = '';
@@ -365,7 +367,7 @@ HTML;
}
$downloads .= <<<HTML
{$a}<br>
<a href="{$a->herf}">{$a->plaintext}</a><br>
HTML;
}
@@ -375,6 +377,9 @@ HTML;
EOD;
}
/**
* Ellipsis title to first 100 characters
*/
private function ellipsisTitle($text) {
$length = 100;
@@ -384,4 +389,15 @@ EOD;
}
return $text;
}
/**
* Strip SVG tag
*/
private function stripSvg($html) {
if ($html->find('svg', 0)) {
$html->find('svg', 0)->outertext = '';
}
return $html;
}
}

View File

@@ -3,13 +3,15 @@ class ComicsKingdomBridge extends BridgeAbstract {
const MAINTAINER = 'stjohnjohnson';
const NAME = 'Comics Kingdom Unofficial RSS';
const URI = 'https://www.comicskingdom.com/';
const URI = 'https://comicskingdom.com/';
const CACHE_TIMEOUT = 21600; // 6h
const DESCRIPTION = 'Comics Kingdom Unofficial RSS';
const PARAMETERS = array( array(
'comicname' => array(
'name' => 'comicname',
'type' => 'text',
'exampleValue' => 'mutts',
'title' => 'The name of the comic in the URL after https://comicskingdom.com/',
'required' => true
)
));
@@ -21,15 +23,13 @@ class ComicsKingdomBridge extends BridgeAbstract {
$author = $html->find('div.author p', 0);;
// Get current date/link
$link = $html->find('meta[property=og:url]', 0)->content;
for($i = 0; $i < 5; $i++) {
$link = $html->find('meta[property=og:url]', -1)->content;
for($i = 0; $i < 3; $i++) {
$item = array();
$page = getSimpleHTMLDOM($link);
$imagelink = $page->find('meta[property=og:image]', 0)->content;
$prevSlug = $page->find('slider-arrow[:is-left-arrow=true]', 0);
$link = $this->getURI() . '/' . $prevSlug->getAttribute('date-slug');
$date = explode('/', $link);
@@ -41,6 +41,8 @@ class ComicsKingdomBridge extends BridgeAbstract {
$item['content'] = '<img src="' . $imagelink . '" />';
$this->items[] = $item;
$link = $page->find('div.comic-viewer-inline a', 0)->href;
if (empty($link)) break; // allow bridge to continue if there's less than 3 comics
}
}

View File

@@ -1,95 +0,0 @@
<?php
class ContainerLinuxReleasesBridge extends BridgeAbstract {
const MAINTAINER = 'captn3m0';
const NAME = 'Core OS Container Linux Releases Bridge';
const URI = 'https://coreos.com/releases/';
const DESCRIPTION = 'Returns the releases notes for Container Linux';
const STABLE = 'stable';
const BETA = 'beta';
const ALPHA = 'alpha';
const PARAMETERS = array(
array(
'channel' => array(
'name' => 'Release Channel',
'type' => 'list',
'defaultValue' => self::STABLE,
'values' => array(
'Stable' => self::STABLE,
'Beta' => self::BETA,
'Alpha' => self::ALPHA,
),
)
)
);
private function getReleaseFeed($jsonUrl) {
$json = getContents($jsonUrl);
return json_decode($json, true);
}
public function getIcon() {
return 'https://coreos.com/assets/ico/favicon.png';
}
public function collectData() {
$data = $this->getReleaseFeed($this->getJsonUri());
foreach ($data as $releaseVersion => $release) {
$item = array();
$item['uri'] = "https://coreos.com/releases/#$releaseVersion";
$item['title'] = $releaseVersion;
$content = $release['release_notes'];
$content .= <<<EOT
Major Software:
* Kernel: {$release['major_software']['kernel'][0]}
* Docker: {$release['major_software']['docker'][0]}
* etcd: {$release['major_software']['etcd'][0]}
EOT;
$item['timestamp'] = strtotime($release['release_date']);
// Based on https://gist.github.com/jbroadway/2836900
// Links
$regex = '/\[([^\[]+)\]\(([^\)]+)\)/';
$replacement = '<a href=\'\2\'>\1</a>';
$item['content'] = preg_replace($regex, $replacement, $content);
// Headings
$regex = '/^(.*)\:\s?$/m';
$replacement = '<h3>\1</h3>';
$item['content'] = preg_replace($regex, $replacement, $item['content']);
// Lists
$regex = '/\n\s*[\*|\-](.*)/';
$item['content'] = preg_replace_callback ($regex, function($regs) {
$item = $regs[1];
return sprintf ('<ul><li>%s</li></ul>', trim ($item));
}, $item['content']);
$this->items[] = $item;
}
}
private function getJsonUri() {
$channel = $this->getInput('channel');
return "https://coreos.com/releases/releases-$channel.json";
}
public function getURI() {
return self::URI;
}
public function getName(){
if(!is_null($this->getInput('channel'))) {
return 'Container Linux Releases: ' . $this->getInput('channel') . ' Channel';
}
return parent::getName();
}
}

View File

@@ -15,11 +15,10 @@ class CourrierInternationalBridge extends FeedExpander {
$item = parent::parseItem($feedItem);
$articlePage = getSimpleHTMLDOMCached($feedItem->link);
$content = $articlePage->find('.article-text', 0);
if(!$content) {
$content = $articlePage->find('.depeche-text', 0);
$content = $articlePage->find('.article-text, depeche-text', 0);
if (!$content) {
return $item;
}
$item['content'] = sanitize($content);
return $item;

View File

@@ -0,0 +1,107 @@
<?php
class CraigslistBridge extends BridgeAbstract {
const MAINTAINER = 'Yaman Qalieh';
const NAME = 'Craigslist Bridge';
const URI = 'https://craigslist.org/';
const DESCRIPTION = 'Returns craigslist search results';
const PARAMETERS = array( array(
'region' => array(
'name' => 'Region',
'title' => 'The subdomain before craigslist.org in the URL',
'exampleValue' => 'sfbay',
'required' => true
),
'search' => array(
'name' => 'Search Query',
'title' => 'Everything in the URL after /search/',
'exampleValue' => 'sya?query=laptop',
'required' => true
),
'limit' => array(
'name' => 'Number of Posts',
'type' => 'number',
'title' => 'The maximum number of posts is 120. Use 0 for unlimited posts.',
'defaultValue' => '25'
)
));
const TEST_DETECT_PARAMETERS = array(
'https://sfbay.craigslist.org/search/sya?query=laptop' => array(
'region' => 'sfbay', 'search' => 'sya?query=laptop'
),
'https://newyork.craigslist.org/search/sss?query=32gb+flash+drive&bundleDuplicates=1&max_price=20' => array(
'region' => 'newyork', 'search' => 'sss?query=32gb+flash+drive&bundleDuplicates=1&max_price=20'
),
);
const URL_REGEX = '/^https:\/\/(?<region>\w+).craigslist.org\/search\/(?<search>.+)/';
public function detectParameters($url) {
if(preg_match(self::URL_REGEX, $url, $matches)) {
$params = array();
$params['region'] = $matches['region'];
$params['search'] = $matches['search'];
return $params;
}
}
public function getURI() {
if (!is_null($this->getInput('region'))) {
$domain = 'https://' . $this->getInput('region') . '.craigslist.org/search/';
return urljoin($domain, $this->getInput('search'));
}
return parent::getURI();
}
public function collectData() {
$uri = $this->getURI();
$html = getSimpleHTMLDOM($uri);
// Check if no results page is shown (nearby results)
if ($html->find('.displaycountShow', 0)->plaintext == '0') {
return;
}
// Search for "more from nearby areas" banner in order to skip those results
$results = $html->find('.result-row, h4.nearby');
// Limit the number of posts
if ($this->getInput('limit') > 0) {
$results = array_slice($results, 0, $this->getInput('limit'));
}
foreach($results as $post) {
// Skip "nearby results" banner and results
// This only appears when searchNearby is not specified
if ($post->tag == 'h4') {
break;
}
$item = array();
$heading = $post->find('.result-heading a', 0);
$item['uri'] = $heading->href;
$item['title'] = $heading->plaintext;
$item['timestamp'] = $post->find('.result-date', 0)->datetime;
$item['uid'] = $heading->id;
$item['content'] = $post->find('.result-price', 0)->plaintext . ' '
// Find the location (local and nearby results if searchNearby=1)
. $post->find('.result-hood, span.nearby', 0)->plaintext;
$images = $post->find('.result-image[data-ids]', 0);
if (!is_null($images)) {
$item['content'] .= '<br>';
foreach(explode(',', $images->getAttribute('data-ids')) as $image) {
// Remove leading 3: from each image id
$id = substr($image, 2);
$image_uri = 'https://images.craigslist.org/' . $id . '_300x300.jpg';
$item['content'] .= '<img src="' . $image_uri . '">';
$item['enclosures'][] = $image_uri;
}
}
$this->items[] = $item;
}
}
}

View File

@@ -4,41 +4,41 @@ class CryptomeBridge extends BridgeAbstract {
const MAINTAINER = 'BoboTiG';
const NAME = 'Cryptome';
const URI = 'https://cryptome.org/';
const CACHE_TIMEOUT = 21600; //6h
const CACHE_TIMEOUT = 21600; // 6h
const DESCRIPTION = 'Returns the N most recent documents.';
const PARAMETERS = array( array(
'n' => array(
'name' => 'number of elements',
'type' => 'number',
'defaultValue' => 20,
'required' => true,
'exampleValue' => 10
)
));
public function getIcon() {
return self::URI . '/favicon.ico';
}
public function collectData(){
$html = getSimpleHTMLDOM(self::URI);
$number = $this->getInput('n');
/* number of documents */
if(!empty($number)) {
$num = min($number, 20);
}
foreach($html->find('pre') as $element) {
for($i = 0; $i < $num; ++$i) {
$i = 0;
foreach($html->find('pre', 1)->find('b') as $element) {
foreach($element->find('a') as $element1) {
$item = array();
$item['uri'] = self::URI . substr($element->find('a', $i)->href, 20);
$item['title'] = substr($element->find('b', $i)->plaintext, 22);
$item['content'] = preg_replace(
'#http://cryptome.org/#',
self::URI,
$element->find('b', $i)->innertext
);
$item['uri'] = $element1->href;
$item['title'] = $element->plaintext;
$this->items[] = $item;
if ($i > $num) {
break 2;
}
$i++;
}
break;
}
}
}

View File

@@ -11,19 +11,22 @@ class DailymotionBridge extends BridgeAbstract {
'By username' => array(
'u' => array(
'name' => 'username',
'required' => true
'required' => true,
'exampleValue' => 'moviepilot',
)
),
'By playlist id' => array(
'p' => array(
'name' => 'playlist id',
'required' => true
'required' => true,
'exampleValue' => 'x6xyc6',
)
),
'From search results' => array(
's' => array(
'name' => 'Search keyword',
'required' => true
'required' => true,
'exampleValue' => 'matrix',
),
'pa' => array(
'name' => 'Page',

View File

@@ -44,92 +44,23 @@ class DanbooruBridge extends BridgeAbstract {
$item['postid'] = (int)preg_replace('/[^0-9]/', '', $element->getAttribute(static::IDATTRIBUTE));
$item['timestamp'] = time();
$thumbnailUri = $element->find('img', 0)->src;
$item['tags'] = $this->getTags($element);
$item['categories'] = array_filter(explode(' ', $this->getTags($element)));
$item['title'] = $this->getName() . ' | ' . $item['postid'];
$item['content'] = '<a href="'
. $item['uri']
. '"><img src="'
. $thumbnailUri
. '" /></a><br>Tags: '
. $item['tags'];
. $this->getTags($element);
return $item;
}
public function collectData(){
$content = getContents($this->getFullURI());
$html = Fix_Simple_Html_Dom::str_get_html($content);
$html = getSimpleHTMLDOMCached($this->getFullURI());
foreach($html->find(static::PATHTODATA) as $element) {
$this->items[] = $this->getItemFromElement($element);
}
}
}
/**
* This class is a monkey patch to 'extend' simplehtmldom to recognize <source>
* tags (HTML5) as self closing tag. This patch should be removed once
* simplehtmldom was fixed. This seems to be a issue with more tags:
* https://sourceforge.net/p/simplehtmldom/bugs/83/
*
* The tag itself is valid according to Mozilla:
*
* The HTML <picture> element serves as a container for zero or more <source>
* elements and one <img> element to provide versions of an image for different
* display device scenarios. The browser will consider each of the child <source>
* elements and select one corresponding to the best match found; if no matches
* are found among the <source> elements, the file specified by the <img>
* element's src attribute is selected. The selected image is then presented in
* the space occupied by the <img> element.
*
* -- https://developer.mozilla.org/en-US/docs/Web/HTML/Element/picture
*
* Notice: This class uses parts of the original simplehtmldom, adjusted to pass
* the guidelines of RSS-Bridge (formatting)
*/
final class Fix_Simple_Html_Dom extends simple_html_dom {
/* copy from simple_html_dom, added 'source' at the end */
protected $self_closing_tags = array(
'img' => 1,
'br' => 1,
'input' => 1,
'meta' => 1,
'link' => 1,
'hr' => 1,
'base' => 1,
'embed' => 1,
'spacer' => 1,
'source' => 1
);
/* copy from simplehtmldom, changed 'simple_html_dom' to 'Fix_Simple_Html_Dom' */
public static function str_get_html($str,
$lowercase = true,
$forceTagsClosed = true,
$target_charset = DEFAULT_TARGET_CHARSET,
$stripRN = true,
$defaultBRText = DEFAULT_BR_TEXT,
$defaultSpanText = DEFAULT_SPAN_TEXT)
{
$dom = new Fix_Simple_Html_Dom(null,
$lowercase,
$forceTagsClosed,
$target_charset,
$stripRN,
$defaultBRText,
$defaultSpanText);
if (empty($str) || strlen($str) > MAX_FILE_SIZE) {
$dom->clear();
return false;
}
$dom->load($str, $lowercase, $stripRN);
return $dom;
}
}

View File

@@ -1,23 +0,0 @@
<?php
class DaveRamseyBlogBridge extends BridgeAbstract {
const MAINTAINER = 'johnpc';
const NAME = 'Dave Ramsey Blog';
const URI = 'https://www.daveramsey.com/blog';
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = 'Returns blog posts from daveramsey.com';
public function collectData()
{
$html = getSimpleHTMLDOM(self::URI);
foreach ($html->find('.Post') as $element) {
$this->items[] = array(
'uri' => 'https://www.daveramsey.com' . $element->find('header > a', 0)->href,
'title' => $element->find('header > h2 > a', 0)->plaintext,
'tags' => $element->find('.Post-topic', 0)->plaintext,
'content' => $element->find('.Post-body', 0)->plaintext,
);
}
}
}

View File

@@ -9,7 +9,14 @@ class DavesTrailerPageBridge extends BridgeAbstract {
$html = getSimpleHTMLDOM(static::URI)
or returnClientError('No results for this query.');
foreach ($html->find('tr[!align]') as $tr) {
$curr_date = null;
foreach ($html->find('tr') as $tr) {
// If it's a date row, update the current date
if ($tr->align == 'center') {
$curr_date = $tr->plaintext;
continue;
}
$item = array();
// title
@@ -21,6 +28,9 @@ class DavesTrailerPageBridge extends BridgeAbstract {
// uri
$item['uri'] = $tr->find('a', 3)->getAttribute('href');
// date: parsed by FeedItem using strtotime
$item['timestamp'] = $curr_date;
$this->items[] = $item;
}
}

View File

@@ -10,6 +10,7 @@ class DealabsBridge extends PepperBridgeAbstract {
'q' => array(
'name' => 'Mot(s) clé(s)',
'type' => 'text',
'exampleValue' => 'lamp',
'required' => true
),
'hide_expired' => array(

View File

@@ -24,7 +24,8 @@ class DerpibooruBridge extends BridgeAbstract {
),
'q' => array(
'name' => 'Query',
'required' => true
'required' => true,
'exampleValue' => 'dog',
)
)
);

View File

@@ -16,7 +16,7 @@ class DesoutterBridge extends BridgeAbstract {
'name' => 'Language',
'type' => 'list',
'title' => 'Select your language',
'defaultValue' => 'Corporate',
'defaultValue' => 'https://www.desouttertools.com/about-desoutter/news-events',
'values' => array(
'Corporate'
=> 'https://www.desouttertools.com/about-desoutter/news-events',
@@ -120,6 +120,7 @@ class DesoutterBridge extends BridgeAbstract {
'limit' => array(
'name' => 'Limit',
'type' => 'number',
'required' => true,
'defaultValue' => 3,
'title' => "Maximum number of items to return in the feed.\n0 = unlimited"
)

View File

@@ -8,6 +8,7 @@ class DiarioDeNoticiasBridge extends BridgeAbstract {
'Tag' => array(
'n' => array(
'name' => 'Tag Name',
'required' => true,
'exampleValue' => 'rogerio-casanova',
)
)

View File

@@ -11,18 +11,26 @@ class DiscogsBridge extends BridgeAbstract {
'artistid' => array(
'name' => 'Artist ID',
'type' => 'number',
'required' => true,
'exampleValue' => '28104',
'title' => 'Only the ID from an artist page. EG /artist/28104-Aesop-Rock is 28104'
)
),
'Label Releases' => array(
'labelid' => array(
'name' => 'Label ID',
'type' => 'number',
'required' => true,
'exampleValue' => '8201',
'title' => 'Only the ID from a label page. EG /label/8201-Rhymesayers-Entertainment is 8201'
)
),
'User Wantlist' => array(
'username_wantlist' => array(
'name' => 'Username',
'type' => 'text',
'required' => true,
'exampleValue' => 'TheBlindMaster',
)
),
'User Folder' => array(

View File

@@ -23,6 +23,7 @@ class DonnonsBridge extends BridgeAbstract {
'p' => array(
'name' => 'Nombre de pages à scanner',
'type' => 'number',
'required' => true,
'defaultValue' => 5,
'title' => 'Indique le nombre de pages de donnons.org qui seront scannées'
)
@@ -66,7 +67,9 @@ class DonnonsBridge extends BridgeAbstract {
$region = $json['availableAtOrFrom']['address']['addressRegion'];
// Grab info from HTML
$imageSrc = $element->find('img.ima-center', 0)->getAttribute('data-src');
$imageSrc = $element->find('img.ima-center', 0)->getAttribute('src');
// Use large image instead of small one
$imageSrc = str_replace('/xs/', '/lg/', $imageSrc);
$image = self::URI . $imageSrc;
$author = $element->find('div.avatar-holder', 0)->plaintext;

View File

@@ -1,194 +0,0 @@
<?php
class DownDetectorBridge extends BridgeAbstract {
const MAINTAINER = 'teromene';
const NAME = 'DownDetector Bridge';
const URI = 'https://downdetector.com/';
const DESCRIPTION = 'Returns most recent downtimes from DownDetector';
const CACHE_TIMEOUT = 300; // 5 min
const PARAMETERS = array(
'All Websites' => array(
'country' => array(
'type' => 'list',
'name' => 'Country',
'values' => array(
'Argentina' => 'https://downdetector.com.ar',
'Australia' => 'https://downdetector.com.au',
'België' => 'https://allestoringen.be',
'Brasil' => 'https://downdetector.com.br',
'Canada' => 'https://downdetector.ca',
'Chile' => 'https://downdetector.cl',
'Colombia' => 'https://downdetector.com.co',
'Danmark' => 'https://downdetector.dk',
'Deutschland' => 'https://allestörungen.de',
'Ecuador' => 'https://downdetector.ec',
'España' => 'https://downdetector.es',
'France' => 'https://downdetector.fr',
'Hong Kong' => 'https://downdetector.hk',
'Hrvatska' => 'https://downdetector.hr',
'India' => 'https://downdetector.in',
'Indonesia' => 'https://downdetector.id',
'Ireland' => 'https://downdetector.ie',
'Italia' => 'https://downdetector.it',
'Magyarország' => 'https://downdetector.hu',
'Malaysia' => 'https://downdetector.my',
'México' => 'https://downdetector.mx',
'Nederland' => 'https://allestoringen.nl',
'New Zealand' => 'https://downdetector.co.nz',
'Norge' => 'https://downdetector.no',
'Pakistan' => 'https://downdetector.pk',
'Perú' => 'https://downdetector.pe',
'Pilipinas' => 'https://downdetector.ph',
'Polska' => 'https://downdetector.pl',
'Portugal' => 'https://downdetector.pt',
'România' => 'https://downdetector.ro',
'Schweiz' => 'https://allestörungen.ch',
'Singapore' => 'https://downdetector.sg',
'Slovensko' => 'https://downdetector.sk',
'South Africa' => 'https://downdetector.co.za',
'Suomi' => 'https://downdetector.fi',
'Sverige' => 'https://downdetector.se',
'Türkiye' => 'https://downdetector.web.tr',
'UAE' => 'https://downdetector.ae',
'UK' => 'https://downdetector.co.uk',
'United States' => 'https://downdetector.com',
'Österreich' => 'https://allestörungen.at',
'Česko' => 'https://downdetector.cz',
'Ελλάς' => 'https://downdetector.gr',
'Россия' => 'https://downdetector.ru',
'日本' => 'https://downdetector.jp'
)
)
),
'Specific Website' => array(
'page' => array(
'type' => 'text',
'name' => 'Status page',
'required' => true,
'exampleValue' => 'https://downdetector.com/status/rainbow-six',
'title' => 'URL of a DownDetector status page e.g: https://downdetector.com/status/rainbow-six/',
)
),
);
private $hostname = '';
private $statusPageId = '';
private $feedname = '';
private $statusUrlRegex = '/\/([a-zA-z0-9ö.]+)\/(?:statu(?:s|t)|problemas?|nu-merge
|(?:feil-)?problem(y|i)?(?:-storningar)?(?:-fejl)?|stoerung|durum|storing|fora-do-ar|ne-rabotaet
|masalah|shougai|ei-toimi)\/([a-zA-Z0-9-]+)/';
public function collectData(){
if ($this->queriedContext == 'Specific Website') {
preg_match($this->statusUrlRegex, $this->getInput('page'), $match)
or returnClientError('Given URL does not seem to at a DownDetector status page!');
$this->hostname = $match[1];
$this->statusPageId = $match[3];
}
$html = getSimpleHTMLDOM($this->getURI() . '/archive/')
or returnClientError('Could not request website!.');
$html = defaultLinkTo($html, $this->getURI());
if ($this->getInput('page')) {
$this->feedname = $html->find('li.breadcrumb-item.active', 0)->plaintext;
}
$table = $html->find('table.table-striped', 0);
if ($table) {
foreach ($table->find('tr') as $event) {
$td = $event->find('td', 0);
if (is_null($td)) {
continue;
}
$item['uri'] = $event->find('td', 0)->find('a', 0)->href;
$item['title'] = $event->find('td', 0)->find('a', 0)->plaintext .
'(' . trim($event->find('td', 1)->plaintext) . ' ' . trim($event->find('td', 2)->plaintext) . ')';
$item['content'] = 'User reports indicate problems at' . $event->find('td', 0)->find('a', 0)->plaintext .
' since ' . $event->find('td', 2)->plaintext;
$item['timestamp'] = $this->formatDate(
trim($event->find('td', 1)->plaintext),
trim($event->find('td', 2)->plaintext)
);
$this->items[] = $item;
}
}
}
public function getURI() {
if($this->getInput('country')) {
return $this->getInput('country');
}
if ($this->getInput('page')) {
return 'https://' . $this->hostname . '/status/' . $this->statusPageId;
}
return self::URI;
}
public function getName() {
if($this->getInput('country')) {
$country = $this->getCountry($this->getInput('country'));
return $country . ' - DownDetector';
}
if ($this->getInput('page')) {
$country = $this->getCountry($this->hostname);
return $this->feedname . ' - ' . $country . ' - DownDetector';
}
return self::NAME;
}
private function formatDate($date, $time) {
switch($this->getCountry()) {
case 'Australia':
case 'UK':
$date = DateTime::createFromFormat('d/m/Y', $date);
return $date->format('Y-m-d') . $time;
case 'Brasil':
case 'Chile':
case 'Colombia':
case 'Ecuador':
case 'España':
case 'Italia':
case 'Perú':
case 'Portugal':
$date = DateTime::createFromFormat('d/m/Y', $date);
return $date->format('Y-m-d') . $time;
case 'Magyarország':
$date = DateTime::createFromFormat('Y.m.d.', $date);
return $date->format('Y-m-d') . $time;
default:
return $date . $time;
}
}
private function getCountry() {
if($this->getInput('country')) {
$input = $this->getInput('country');
}
if ($this->getInput('page')) {
if (empty($this->hostname)) {
return 'N/A';
}
$input = 'https://' . $this->hostname;
}
$parameters = $this->getParameters();
$countryValues = array_flip($parameters['All Websites']['country']['values']);
$country = $countryValues[$input];
return $country;
}
}

View File

@@ -13,6 +13,7 @@ class DuckDuckGoBridge extends BridgeAbstract {
const PARAMETERS = array( array(
'u' => array(
'name' => 'keyword',
'exampleValue' => 'duck',
'required' => true
),
'sort' => array(

View File

@@ -1,159 +0,0 @@
<?php
class ETTVBridge extends BridgeAbstract {
const MAINTAINER = 'GregThib';
const NAME = 'ETTV';
const URI = 'https://www.ettv.tv/';
const DESCRIPTION = 'Returns list of 20 latest torrents for a specific search.';
const CACHE_TIMEOUT = 14400; // 4 hours
const PARAMETERS = array( array(
'query' => array(
'name' => 'Keywords',
'required' => true
),
'cat' => array(
'type' => 'list',
'name' => 'Category',
'values' => array(
'(ALL TYPES)' => '0',
'Anime: Movies' => '73',
'Anime: Dubbed/Subbed' => '74',
'Anime: Others' => '75',
'Books: Ebooks' => '53',
'Books: Magazines' => '54',
'Books: Comics' => '55',
'Books: Audio' => '56',
'Books: Others' => '68',
'Games: Windows' => '57',
'Games: Android' => '58',
'Games: Others' => '71',
'Movies: HD 1080p' => '1',
'Movies: HD 720p' => '2',
'Movies: UltraHD/4K' => '3',
'Movies: XviD' => '42',
'Movies: X264/H264' => '47',
'Movies: 3D' => '49',
'Movies: Dubs/Dual Audio' => '51',
'Movies: CAM/TS' => '65',
'Movies: BluRay Disc/Remux' => '66',
'Movies: DVDR' => '67',
'Movies: HEVC/x265' => '76',
'Music: MP3' => '59',
'Music: FLAC' => '60',
'Music: Music Videos' => '61',
'Music: Others' => '69',
'Software: Windows' => '62',
'Software: Android' => '63',
'Software: Mac' => '64',
'Software: Others' => '70',
'TV: HD/X264/H264' => '41',
'TV: SD/X264/H264' => '5',
'TV: TV Packs' => '7',
'TV: SD/XVID' => '50',
'TV: Sport' => '72',
'TV: HEVC/x265' => '77',
'Unsorted: Unsorted' => '78'
),
'defaultValue' => '(ALL TYPES)'
),
'status' => array(
'type' => 'list',
'name' => 'Status',
'values' => array(
'Active Transfers' => '0',
'Included Dead' => '1',
'Only Dead' => '2'
),
'defaultValue' => 'Included Dead'
),
'lang' => array(
'type' => 'list',
'name' => 'Lang',
'values' => array(
'(ALL)' => '0',
'Arabic' => '17',
'Chinese ' => '10',
'Danish' => '13',
'Dutch' => '11',
'English' => '1',
'Finnish' => '18',
'French' => '2',
'German' => '3',
'Greek' => '15',
'Hindi' => '8',
'Italian' => '4',
'Japanese' => '5',
'Korean' => '9',
'Polish' => '14',
'Russian' => '7',
'Spanish' => '6',
'Turkish' => '16'
),
'defaultValue' => '(ALL)'
)
));
protected $results_link;
public function collectData(){
// No control on inputs, because all defaultValue are set
$query_str = 'torrents-search.php';
$query_str .= '?search=' . urlencode('+' . str_replace(' ', ' +', $this->getInput('query')));
$query_str .= '&cat=' . $this->getInput('cat');
$query_str .= '&incldead=' . $this->getInput('status');
$query_str .= '&lang=' . $this->getInput('lang');
$query_str .= '&sort=id&order=desc';
// Get results page
$this->results_link = self::URI . $query_str;
$html = getSimpleHTMLDOM($this->results_link);
// Loop on each entry
foreach($html->find('table.table tr') as $element) {
if($element->parent->tag == 'thead') continue;
$entry = $element->find('td', 1)->find('a', 0);
// retrieve result page to get more details
$link = rtrim(self::URI, '/') . $entry->href;
$page = getSimpleHTMLDOM($link);
// get details & download links
$details = $page->find('fieldset.download table', 0); // WHAT?? It should be the second one…
$dllinks = $page->find('div#downloadbox table', 0);
// fill item
$item = array();
$item['author'] = $details->children(6)->children(1)->plaintext;
$item['title'] = $entry->title;
$item['uri'] = $link;
$item['timestamp'] = strtotime($details->children(7)->children(1)->plaintext);
$item['content'] = '';
$item['content'] .= '<br/><b>Name: </b>' . $details->children(0)->children(1)->innertext;
$item['content'] .= '<br/><b>Lang: </b>' . $details->children(3)->children(1)->innertext;
$item['content'] .= '<br/><b>Size: </b>' . $details->children(4)->children(1)->innertext;
$item['content'] .= '<br/><b>Hash: </b>' . $details->children(5)->children(1)->innertext;
foreach($dllinks->children(0)->children(1)->find('a') as $dl) {
$item['content'] .= '<br/>' . $dl->outertext;
}
$item['content'] .= '<br/><br/>' . $details->children(1)->children(0)->innertext;
$this->items[] = $item;
}
}
public function getName(){
if($this->getInput('query')) {
return '[' . self::NAME . '] ' . $this->getInput('query');
}
return self::NAME;
}
public function getURI(){
if(isset($this->results_link) && !empty($this->results_link)) {
return $this->results_link;
}
return self::URI;
}
}

View File

@@ -3,64 +3,109 @@ class EZTVBridge extends BridgeAbstract {
const MAINTAINER = 'alexAubin';
const NAME = 'EZTV';
const URI = 'https://eztv.ch/';
const DESCRIPTION = 'Returns list of *recent* torrents for a specific show
on EZTV. Get showID from URLs in https://eztv.ch/shows/showID/show-full-name.';
const URI = 'https://eztv.re/';
const DESCRIPTION = 'Returns list of torrents for specific show(s)
on EZTV. Get IMDB IDs from IMDB.';
const PARAMETERS = array( array(
'i' => array(
'name' => 'Show ids',
'exampleValue' => 'showID1,showID2,…',
'required' => true
const PARAMETERS = array(
array(
'ids' => array(
'name' => 'Show IMDB IDs',
'exampleValue' => '8740790,1733785',
'required' => true,
'title' => 'One or more IMDB show IDs (can be found in the IMDB show URL)'
),
'no480' => array(
'name' => 'No 480p',
'type' => 'checkbox',
'title' => 'Activate to exclude 480p torrents'
),
'no720' => array(
'name' => 'No 720p',
'type' => 'checkbox',
'title' => 'Activate to exclude 720p torrents'
),
'no1080' => array(
'name' => 'No 1080p',
'type' => 'checkbox',
'title' => 'Activate to exclude 1080p torrents'
),
'no2160' => array(
'name' => 'No 2160p',
'type' => 'checkbox',
'title' => 'Activate to exclude 2160p torrents'
),
'noUnknownRes' => array(
'name' => 'No Unknown resolution',
'type' => 'checkbox',
'title' => 'Activate to exclude unknown resolution torrents'
),
)
));
);
// Shamelessly lifted from https://stackoverflow.com/a/2510459
protected function formatBytes($bytes, $precision = 2) {
$units = array('B', 'KB', 'MB', 'GB', 'TB');
$bytes = max($bytes, 0);
$pow = floor(($bytes ? log($bytes) : 0) / log(1024));
$pow = min($pow, count($units) - 1);
$bytes /= pow(1024, $pow);
return round($bytes, $precision) . ' ' . $units[$pow];
}
protected function getItemFromTorrent($torrent){
$item = array();
$item['uri'] = $torrent->episode_url;
$item['author'] = $torrent->imdb_id;
$item['timestamp'] = date('d F Y H:i:s', $torrent->date_released_unix);
$item['title'] = $torrent->title;
$item['enclosures'][] = $torrent->torrent_url;
$thumbnailUri = 'https:' . $torrent->small_screenshot;
$torrentSize = $this->formatBytes($torrent->size_bytes);
$item['content'] = $torrent->filename . '<br>File size: '
. $torrentSize . '<br><a href="' . $torrent->magnet_url
. '">magnet link</a><br><a href="' . $torrent->torrent_url
. '">torrent link</a><br><img src="' . $thumbnailUri . '" />';
return $item;
}
private static function compareDate($torrent1, $torrent2) {
return (strtotime($torrent1['timestamp']) < strtotime($torrent2['timestamp']) ? 1 : -1);
}
public function collectData(){
$showIds = explode(',', $this->getInput('ids'));
// Make timestamp from relative released time in table
function makeTimestamp($relativeReleaseTime){
foreach($showIds as $showId) {
$eztvUri = $this->getURI() . 'api/get-torrents?imdb_id=' . $showId;
$content = getContents($eztvUri);
$torrents = json_decode($content)->torrents;
foreach($torrents as $torrent) {
$title = $torrent->title;
$regex480 = '/480p/';
$regex720 = '/720p/';
$regex1080 = '/1080p/';
$regex2160 = '/2160p/';
$regexUnknown = '/(480p|720p|1080p|2160p)/';
// Skip unwanted resolution torrents
if ((preg_match($regex480, $title) === 1 && $this->getInput('no480'))
|| (preg_match($regex720, $title) === 1 && $this->getInput('no720'))
|| (preg_match($regex1080, $title) === 1 && $this->getInput('no1080'))
|| (preg_match($regex2160, $title) === 1 && $this->getInput('no2160'))
|| (preg_match($regexUnknown, $title) !== 1 && $this->getInput('noUnknownRes'))) {
continue;
}
$relativeDays = 0;
$relativeHours = 0;
foreach(explode(' ', $relativeReleaseTime) as $relativeTimeElement) {
if(substr($relativeTimeElement, -1) == 'd') $relativeDays = substr($relativeTimeElement, 0, -1);
if(substr($relativeTimeElement, -1) == 'h') $relativeHours = substr($relativeTimeElement, 0, -1);
}
return mktime(date('h') - $relativeHours, 0, 0, date('m'), date('d') - $relativeDays, date('Y'));
}
// Loop on show ids
$showList = explode(',', $this->getInput('i'));
foreach($showList as $showID) {
// Get show page
$html = getSimpleHTMLDOM(self::URI . 'shows/' . rawurlencode($showID) . '/');
// Loop on each element that look like an episode entry...
foreach($html->find('.forum_header_border') as $element) {
// Filter entries that are not episode entries
$ep = $element->find('td', 1);
if(empty($ep)) continue;
$epinfo = $ep->find('.epinfo', 0);
$released = $element->find('td', 3);
if(empty($epinfo)) continue;
if(empty($released->plaintext)) continue;
// Filter entries that are older than 1 week
if($released->plaintext == '&gt;1 week') continue;
// Fill item
$item = array();
$item['uri'] = self::URI . $epinfo->href;
$item['id'] = $item['uri'];
$item['timestamp'] = makeTimestamp($released->plaintext);
$item['title'] = $epinfo->plaintext;
$item['content'] = $epinfo->alt;
if(isset($item['title']))
$this->items[] = $item;
$this->items[] = $this->getItemFromTorrent($torrent);
}
}
// Sort all torrents in array by date
usort($this->items, array('EZTVBridge', 'compareDate'));
}
}

View File

@@ -95,23 +95,38 @@ class EconomistBridge extends FeedExpander {
protected function parseItem($feedItem){
$item = parent::parseItem($feedItem);
$article = getSimpleHTMLDOM($item['uri']);
// before the article can be added, it needs to be cleaned up, thus, the extra function
$item['content'] = $this->cleanContent($article);
// We also need to distinguish between old style and new style articles
if ($article->find('article', 0)->getAttribute('data-test-id') == 'Article') {
$contentNode = 'div.layout-article-body';
$imgNode = 'div.article__lead-image';
$categoryNode = 'span.article__subheadline';
} elseif ($article->find('article', 0)->getAttribute('data-test-id') === 'NewArticle') {
$contentNode = 'section';
$imgNode = 'figure.css-12eysrk.e3y6nua0';
$categoryNode = 'span.ern1uyf0';
} else {
return;
}
$item['content'] = $this->cleanContent($article, $contentNode);
// only the article lead image is retained if it's there
if (!is_null($article->find('div.article__lead-image', 0))) {
$item['enclosures'][] = $article->find('div.article__lead-image', 0)->find('img', 0)->getAttribute('src');
if (!is_null($article->find($imgNode, 0))) {
$item['enclosures'][] = $article->find($imgNode, 0)->find('img', 0)->getAttribute('src');
} else {
$item['enclosures'][] = '';
}
// add the subheadline as category. This will create a link in new articles
// and a text in old articles
$item['categories'][] = $article->find($categoryNode, 0)->innertext;
return $item;
}
private function cleanContent($article){
private function cleanContent($article, $contentNode){
// the actual article is in this div
$content = $article->find('div.layout-article-body', 0)->innertext;
$content = $article->find($contentNode, 0)->innertext;
// clean the article content. Remove all div's since the text is in paragraph elements
foreach (array(
'<div '

View File

@@ -12,6 +12,7 @@ class ElloBridge extends BridgeAbstract {
'u' => array(
'name' => 'Username',
'required' => true,
'exampleValue' => 'zteph',
'title' => 'Username'
)
),
@@ -19,6 +20,7 @@ class ElloBridge extends BridgeAbstract {
's' => array(
'name' => 'Search',
'required' => true,
'exampleValue' => 'bird',
'title' => 'Search'
)
)

View File

@@ -1,7 +1,7 @@
<?php
class ElsevierBridge extends BridgeAbstract {
const MAINTAINER = 'Pierre Mazière';
const MAINTAINER = 'dvikan';
const NAME = 'Elsevier journals recent articles';
const URI = 'https://www.journals.elsevier.com/';
const CACHE_TIMEOUT = 43200; //12h
@@ -11,68 +11,31 @@ class ElsevierBridge extends BridgeAbstract {
'j' => array(
'name' => 'Journal name',
'required' => true,
'exampleValue' => 'academic-pediactrics',
'exampleValue' => 'academic-pediatrics',
'title' => 'Insert html-part of your journal'
)
));
// Extracts the list of names from an article as string
private function extractArticleName($article){
$names = $article->find('small', 0);
if($names)
return trim($names->plaintext);
return '';
}
// Extracts the timestamp from an article
private function extractArticleTimestamp($article){
$time = $article->find('.article-info', 0);
if($time) {
$timestring = trim($time->plaintext);
/*
The format depends on the age of an article:
- Available online 29 July 2016
- July 2016
- MayJune 2016
*/
if(preg_match('/\S*(\d+\s\S+\s\d{4})/ims', $timestring, $matches)) {
return strtotime($matches[0]);
} elseif (preg_match('/[A-Za-z]+\-([A-Za-z]+\s\d{4})/ims', $timestring, $matches)) {
return strtotime($matches[0]);
} elseif (preg_match('/([A-Za-z]+\s\d{4})/ims', $timestring, $matches)) {
return strtotime($matches[0]);
} else {
return 0;
}
}
return 0;
}
// Extracts the content from an article
private function extractArticleContent($article){
$content = $article->find('.article-content', 0);
if($content) {
return trim($content->plaintext);
}
return '';
}
public function getIcon() {
return 'https://cdn.elsevier.io/verona/includes/favicons/favicon-32x32.png';
}
public function collectData(){
$uri = self::URI . $this->getInput('j') . '/recent-articles/';
$html = getSimpleHTMLDOM($uri);
// Not all journals have the /recent-articles page
$url = sprintf('https://www.journals.elsevier.com/%s/recent-articles/', $this->getInput('j'));
$html = getSimpleHTMLDOM($url);
foreach($html->find('.pod-listing') as $article) {
$item = array();
$item['uri'] = $article->find('.pod-listing-header>a', 0)->getAttribute('href') . '?np=y';
$item['title'] = $article->find('.pod-listing-header>a', 0)->plaintext;
$item['author'] = $this->extractArticleName($article);
$item['timestamp'] = $this->extractArticleTimestamp($article);
$item['content'] = $this->extractArticleContent($article);
foreach($html->find('article') as $recentArticle) {
$item = [];
$item['uri'] = $recentArticle->find('a', 0)->getAttribute('href');
$item['title'] = $recentArticle->find('h2', 0)->plaintext;
$item['author'] = $recentArticle->find('p > span', 0)->plaintext;
$publicationDateString = trim($recentArticle->find('p > span', 1)->plaintext);
$publicationDate = DateTimeImmutable::createFromFormat('F d, Y', $publicationDateString);
if ($publicationDate) {
$item['timestamp'] = $publicationDate->getTimestamp();
}
$this->items[] = $item;
}
}
public function getIcon(): string {
return 'https://cdn.elsevier.io/verona/includes/favicons/favicon-32x32.png';
}
}

View File

@@ -11,6 +11,7 @@ class EpicgamesBridge extends BridgeAbstract {
'postcount' => array(
'name' => 'Limit',
'type' => 'number',
'required' => true,
'title' => 'Maximum number of items to return',
'defaultValue' => 10,
),

View File

@@ -12,7 +12,7 @@ class EtsyBridge extends BridgeAbstract {
'type' => 'text',
'required' => true,
'title' => 'Insert your search term here',
'exampleValue' => 'Enter your search term'
'exampleValue' => 'lamp'
),
'queryextension' => array(
'name' => 'Query extension',
@@ -22,12 +22,10 @@ class EtsyBridge extends BridgeAbstract {
(anything after ?search=<your search query>)',
'exampleValue' => '&explicit=1&locationQuery=2921044'
),
'showimage' => array(
'name' => 'Show image in content',
'hideimage' => array(
'name' => 'Hide image in content',
'type' => 'checkbox',
'required' => false,
'title' => 'Activate to show the image in the content',
'defaultValue' => 'checked'
'title' => 'Activate to hide the image in the content',
)
)
);
@@ -35,29 +33,29 @@ class EtsyBridge extends BridgeAbstract {
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI());
$results = $html->find('li.block-grid-item');
$results = $html->find('li.wt-list-unstyled');
foreach($results as $result) {
// Skip banner cards (ads for categories)
if($result->find('span.ad-indicator'))
// Remove Lazy loading
if($result->find('.wt-skeleton-ui', 0))
continue;
$item = array();
$item['title'] = $result->find('a', 0)->title;
$item['uri'] = $result->find('a', 0)->href;
$item['author'] = $result->find('p.text-gray-lighter', 0)->plaintext;
$item['author'] = $result->find('p.wt-text-gray > span', 2)->plaintext;
$item['content'] = '<p>'
. $result->find('span.currency-value', 0)->plaintext . ' '
. $result->find('span.currency-symbol', 0)->plaintext
. $result->find('span.currency-value', 0)->plaintext
. '</p><p>'
. $result->find('a', 0)->title
. '</p>';
$image = $result->find('img.display-block', 0)->src;
$image = $result->find('img.wt-display-block', 0)->src;
if($this->getInput('showimage')) {
if(!$this->getInput('hideimage')) {
$item['content'] .= '<img src="' . $image . '">';
}

View File

@@ -0,0 +1,38 @@
<?php
class ExecuteProgramBridge extends BridgeAbstract
{
const NAME = 'Execute Program Blog';
const URI = 'https://www.executeprogram.com/blog';
const DESCRIPTION = 'Unofficial feed for the www.executeprogram.com blog';
const MAINTAINER = 'dvikan';
public function collectData()
{
$data = json_decode(getContents('https://www.executeprogram.com/api/pages/blog'));
foreach ($data->posts as $post) {
$year = $post->date->year;
$month = $post->date->month;
$day = $post->date->day;
$item = array();
$item['uri'] = sprintf('https://www.executeprogram.com/blog/%s', $post->slug);
$item['title'] = $post->title;
$dateTime = \DateTime::createFromFormat('Y-m-d', $year . '-' . $month . '-' . $day);
$item['timestamp'] = $dateTime->format('U');
$item['content'] = $post->body;
$this->items[] = $item;
}
usort($this->items, function ($a, $b) {
return $a['timestamp'] < $b['timestamp'];
});
}
public function getIcon()
{
return 'https://www.executeprogram.com/favicon.ico';
}
}

View File

@@ -229,7 +229,7 @@ EOD
$ctx = stream_context_create(array(
'http' => array(
'user_agent' => 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0',
'user_agent' => Configuration::getConfig('http', 'useragent'),
'Accept' => 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
)
)
@@ -254,7 +254,7 @@ EOD
$context = stream_context_create(array(
'http' => array(
'user_agent' => 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0',
'user_agent' => Configuration::getConfig('http', 'useragent'),
'header' => 'Cookie: ' . $cookies
)
)

View File

@@ -4,7 +4,7 @@ class FDroidBridge extends BridgeAbstract {
const MAINTAINER = 'Mitsukarenai';
const NAME = 'F-Droid Bridge';
const URI = 'https://f-droid.org/';
const CACHE_TIMEOUT = 60 * 60 * 2; // 2 hours
const CACHE_TIMEOUT = 60 * 60 * 4; // 4 hours
const DESCRIPTION = 'Returns latest added/updated apps on the open-source Android apps repository F-Droid';
const PARAMETERS = array( array(
@@ -22,6 +22,32 @@ class FDroidBridge extends BridgeAbstract {
return self::URI . 'assets/favicon.ico?v=8j6PKzW9Mk';
}
private function getTimestamp($url) {
$curlOptions = array(
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HEADER => true,
CURLOPT_NOBODY => true,
CURLOPT_CONNECTTIMEOUT => 19,
CURLOPT_TIMEOUT => 19,
);
$ch = curl_init($url);
curl_setopt_array($ch, $curlOptions);
$curlHeaders = curl_exec($ch);
$curlError = curl_error($ch);
curl_close($ch);
if(!empty($curlError))
return false;
$curlHeaders = explode("\n", $curlHeaders);
$timestamp = false;
foreach($curlHeaders as $header) {
if(strpos($header, 'Last-Modified') !== false) {
$timestamp = str_replace('Last-Modified: ', '', $header);
$timestamp = strtotime($timestamp);
}
}
return $timestamp;
}
public function collectData(){
$url = self::URI;
$html = getSimpleHTMLDOM($url);
@@ -45,6 +71,7 @@ class FDroidBridge extends BridgeAbstract {
$item['uri'] = self::URI . $element->href;
$item['title'] = $element->find('h4', 0)->plaintext;
$item['icon'] = $element->find('img', 0)->src;
$item['timestamp'] = $this->getTimestamp($item['icon']);
$item['summary'] = $element->find('span.package-summary', 0)->plaintext;
$item['content'] = '
<a href="' . $item['uri'] . '">

View File

@@ -13,6 +13,7 @@ class FSecureBlogBridge extends BridgeAbstract {
),
'language' => array(
'name' => 'Language',
'required' => true,
'defaultValue' => 'en',
),
'oldest_date' => array(

View File

@@ -0,0 +1,54 @@
<?php
class FeedMergeBridge extends FeedExpander {
const MAINTAINER = 'dvikan';
const NAME = 'FeedMerge';
const URI = 'https://github.com/RSS-Bridge/rss-bridge';
const DESCRIPTION = <<<'TEXT'
This bridge merges two or more feeds into a single feed. Max 10 items are fetched from each feed.
TEXT;
const PARAMETERS = [
[
'feed_name' => [
'name' => 'Feed name',
'type' => 'text',
'exampleValue' => 'rss-bridge/FeedMerger',
],
'feed_1' => [
'name' => 'Feed url',
'type' => 'text',
'required' => true,
'exampleValue' => 'https://lorem-rss.herokuapp.com/feed?unit=day'
],
'feed_2' => ['name' => 'Feed url', 'type' => 'text'],
'feed_3' => ['name' => 'Feed url', 'type' => 'text'],
'feed_4' => ['name' => 'Feed url', 'type' => 'text'],
'feed_5' => ['name' => 'Feed url', 'type' => 'text'],
]
];
public function collectData() {
$limit = 10;
$feeds = [
$this->getInput('feed_1'),
$this->getInput('feed_2'),
$this->getInput('feed_3'),
$this->getInput('feed_4'),
$this->getInput('feed_5'),
];
// Remove empty values
$feeds = array_filter($feeds);
foreach ($feeds as $feed) {
$this->collectExpandableDatas($feed, $limit);
}
}
public function getIcon() {
return 'https://cdn.jsdelivr.net/npm/famfamfam-silk@1.0.0/dist/png/folder_feed.png';
}
public function getName() {
return $this->getInput('feed_name') ?: 'rss-bridge/FeedMerger';
}
}

View File

@@ -0,0 +1,60 @@
<?php
class FeedReducerBridge extends FeedExpander {
const MAINTAINER = 'mdemoss';
const NAME = 'Feed Reducer';
const URI = 'http://github.com/RSS-Bridge/rss-bridge/';
const DESCRIPTION = 'Choose a percentage of a feed you want to see.';
const PARAMETERS = array( array(
'url' => array(
'name' => 'Feed URI',
'exampleValue' => 'https://lorem-rss.herokuapp.com/feed?length=42',
'required' => true
),
'percentage' => array(
'name' => 'percentage',
'type' => 'number',
'exampleValue' => 50,
'required' => true
)
));
const CACHE_TIMEOUT = 3600;
public function collectData(){
if(preg_match('#^http(s?)://#i', $this->getInput('url'))) {
$this->collectExpandableDatas($this->getInput('url'));
} else {
throw new Exception('URI must begin with http(s)://');
}
}
public function getItems(){
$filteredItems = array();
$intPercentage = (int)preg_replace('/[^0-9]/', '', $this->getInput('percentage'));
foreach ($this->items as $thisItem) {
// The URL is included in the hash:
// - so you can change the output by adding a local-part to the URL
// - so items with the same URI in different feeds won't be correlated
// $pseudoRandomInteger will be a 16 bit unsigned int mod 100.
// This won't be uniformly distributed 1-100, but should be close enough.
$pseudoRandomInteger = unpack(
'S', // unsigned 16-bit int
hash( 'sha256', $thisItem['uri'] . '::' . $this->getInput('url'), true )
)[1] % 100;
if ($pseudoRandomInteger < $intPercentage) {
$filteredItems[] = $thisItem;
}
}
return $filteredItems;
}
public function getName(){
$trimmedPercentage = preg_replace('/[^0-9]/', '', $this->getInput('percentage') ?? '');
return parent::getName() . ' [' . $trimmedPercentage . '%]';
}
}

View File

@@ -11,6 +11,8 @@ class FilterBridge extends FeedExpander {
const PARAMETERS = array(array(
'url' => array(
'name' => 'Feed URL',
'type' => 'text',
'defaultValue' => 'https://lorem-rss.herokuapp.com/feed?unit=day',
'required' => true,
),
'filter' => array(

View File

@@ -62,11 +62,11 @@ class FindACrewBridge extends BridgeAbstract {
foreach ($annonces as $annonce) {
$item = array();
$link = parent::getURI() . $annonce->find('.lst-ctrls a', 0)->href;
$link = parent::getURI() . $annonce->find('.lstsum-btn-con a', 0)->href;
$htmlDetail = getSimpleHTMLDOMCached($link . '?mdl=2'); // add ?mdl=2 for xhr content not full html page
$img = parent::getURI() . $htmlDetail->find('img.img-responsive', 0)->getAttribute('src');
$item['title'] = $annonce->find('.lst-tags span', 0)->plaintext;
$item['title'] = $htmlDetail->find('div.label-account', 0)->plaintext;
$item['uri'] = $link;
$content = $htmlDetail->find('.panel-body div.clearfix.row > div', 1)->innertext;
$content .= $htmlDetail->find('.panel-body > div', 1)->innertext;

View File

@@ -51,6 +51,15 @@ class FlickrBridge extends BridgeAbstract {
'title' => 'Insert username (as shown in the address bar)',
'exampleValue' => 'flickr'
),
'content' => array(
'name' => 'Content',
'type' => 'list',
'values' => array(
'Uploads' => 'uploads',
'Favorites' => 'faves',
),
'defaultValue' => 'uploads',
),
'media' => array(
'name' => 'Media',
'type' => 'list',
@@ -156,8 +165,14 @@ class FlickrBridge extends BridgeAbstract {
. '&sort=' . $this->getInput('sort') . '&media=' . $this->getInput('media');
break;
case 'By username':
return self::URI . 'search/?user_id=' . urlencode($this->getInput('u'))
. '&sort=' . $this->getInput('sort') . '&media=' . $this->getInput('media');
$uri = self::URI . 'search/?user_id=' . urlencode($this->getInput('u'))
. '&sort=date-posted-desc&media=' . $this->getInput('media');
if ($this->getInput('content') === 'faves') {
return $uri . '&faves=1';
}
return $uri;
break;
default:
@@ -175,6 +190,11 @@ class FlickrBridge extends BridgeAbstract {
return $this->getInput('q') . ' - keyword - ' . self::NAME;
break;
case 'By username':
if ($this->getInput('content') === 'faves') {
return $this->username . ' - favorites - ' . self::NAME;
}
return $this->username . ' - ' . self::NAME;
break;

View File

@@ -9,29 +9,45 @@ class FolhaDeSaoPauloBridge extends FeedExpander {
'feed' => array(
'name' => 'Feed sub-URL',
'type' => 'text',
'required' => true,
'title' => 'Select the sub-feed (see https://www1.folha.uol.com.br/feed/)',
'exampleValue' => 'emcimadahora/rss091.xml',
)
),
'amount' => array(
'name' => 'Amount of items to fetch',
'type' => 'number',
'defaultValue' => 15,
),
'deep_crawl' => array(
'name' => 'Deep Crawl',
'description' => 'Crawl each item "deeply", that is, return the article contents',
'type' => 'checkbox',
'defaultValue' => true,
),
)
);
protected function parseItem($item){
$item = parent::parseItem($item);
$articleHTMLContent = getSimpleHTMLDOMCached($item['uri']);
if($articleHTMLContent) {
foreach ($articleHTMLContent->find('div.c-news__body .is-hidden') as $toRemove) {
$toRemove->innertext = '';
}
$item_content = $articleHTMLContent->find('div.c-news__body', 0);
if ($item_content) {
$text = $item_content->innertext;
$text = strip_tags($text, '<p><b><a><blockquote><figure><figcaption><img><strong><em>');
$item['content'] = $text;
$item['uri'] = explode('*', $item['uri'])[1];
if ($this->getInput('deep_crawl')) {
$articleHTMLContent = getSimpleHTMLDOMCached($item['uri']);
if($articleHTMLContent) {
foreach ($articleHTMLContent->find('div.c-news__body .is-hidden') as $toRemove) {
$toRemove->innertext = '';
}
$item_content = $articleHTMLContent->find('div.c-news__body', 0);
if ($item_content) {
$text = $item_content->innertext;
$text = strip_tags($text, '<p><b><a><blockquote><figure><figcaption><img><strong><em><ul><li>');
$item['content'] = $text;
$item['uri'] = explode('*', $item['uri'])[1];
}
} else {
Debug::log('???: ' . $item['uri']);
}
} else {
Debug::log('???: ' . $item['uri']);
$item['uri'] = explode('*', $item['uri'])[1];
}
return $item;
@@ -47,6 +63,6 @@ class FolhaDeSaoPauloBridge extends FeedExpander {
$feed_url = self::URI . '/' . $this->getInput('feed');
}
Debug::log('URL: ' . $feed_url);
$this->collectExpandableDatas($feed_url);
$this->collectExpandableDatas($feed_url, $this->getInput('amount'));
}
}

View File

@@ -1,74 +0,0 @@
<?php
class FootitoBridge extends BridgeAbstract {
const MAINTAINER = 'superbaillot.net';
const NAME = 'Footito';
const URI = 'http://www.footito.fr/';
const DESCRIPTION = 'Footito';
public function collectData(){
$html = getSimpleHTMLDOM(self::URI);
foreach($html->find('div.post') as $element) {
$item = array();
$content = trim($element->innertext);
$content = str_replace(
'<img',
"<img style='float : left;'",
$content );
$content = str_replace(
'class="logo"',
"style='float : left;'",
$content );
$content = str_replace(
'class="contenu"',
"style='margin-left : 60px;'",
$content );
$content = str_replace(
'class="responsive-comment"',
"style='border-top : 1px #DDD solid; background-color : white; padding : 10px;'",
$content );
$content = str_replace(
'class="jaime"',
"style='display : none;'",
$content );
$content = str_replace(
'class="auteur-event responsive"',
"style='display : none;'",
$content );
$content = str_replace(
'class="report-abuse-button"',
"style='display : none;'",
$content );
$content = str_replace(
'class="reaction clearfix"',
"style='margin : 10px 0px; padding : 5px; border-bottom : 1px #DDD solid;'",
$content );
$content = str_replace(
'class="infos"',
"style='font-size : 0.7em;'",
$content );
$item['content'] = $content;
$title = $element->find('.contenu .texte ', 0)->plaintext;
$item['title'] = $title;
$info = $element->find('div.infos', 0);
$item['timestamp'] = strtotime($info->find('time', 0)->datetime);
$item['author'] = $info->find('a.auteur', 0)->plaintext;
$this->items[] = $item;
}
}
}

View File

@@ -10,11 +10,13 @@ class FourchanBridge extends BridgeAbstract {
const PARAMETERS = array( array(
'c' => array(
'name' => 'Thread category',
'required' => true
'required' => true,
'exampleValue' => 'po',
),
't' => array(
'name' => 'Thread number',
'type' => 'number',
'exampleValue' => '597271',
'required' => true
)
));

84
bridges/FunkBridge.php Normal file
View File

@@ -0,0 +1,84 @@
<?php
class FunkBridge extends BridgeAbstract {
const MAINTAINER = 'µKöff';
const NAME = 'Funk';
const URI = 'https://www.funk.net/';
const DESCRIPTION = 'Videos per channel of German public video-on-demand service Funk';
const PARAMETERS = array(
'Channel' => array(
'channel' => array(
'name' => 'Slug',
'type' => 'text',
'required' => true,
'exampleValue' => 'game-two-856'
),
'max' => array(
'name' => 'Maximum',
'type' => 'number',
'defaultValue' => '-1'
)
)
);
public function collectData(){
switch($this->queriedContext) {
case 'Channel':
$url = static::URI . 'data/videos/byChannelAlias/' . $this->getInput('channel') . '/';
if(!empty($this->getInput('max')) && $this->getInput('max') >= 0) {
$url .= '?size=' . $this->getInput('max');
}
$jsonString = getContents($url) or returnServerError('No contents received!');
$json = json_decode($jsonString, true);
foreach($json['list'] as $element) {
$this->items[] = $this->collectArticle($element);
}
break;
default:
returnServerError('Unknown context!');
}
}
private function collectArticle($element) {
$item = array();
$item['uri'] = static::URI . 'channel/' . $element['channelAlias'] . '/' . $element['alias'];
$item['title'] = $element['title'];
$item['timestamp'] = $element['publicationDate'];
$item['author'] = str_replace('-' . $element['channelId'], '', $element['channelAlias']);
$item['content'] = $element['shortDescription'];
$item['enclosures'] = array(
'https://www.funk.net/api/v4.0/thumbnails/' . $element['imageLandscape']
);
$item['uid'] = $element['entityId'];
return $item;
}
public function detectParameters($url) {
$regex = '/^https?:\/\/(?:www\.)?funk\.net\/channel\/([^\/]+).*$/';
if(preg_match($regex, $url, $urlMatches) > 0) {
return array(
'channel' => $urlMatches[1]
);
} else {
return null;
}
}
public function getIcon() {
return 'https://www.funk.net/img/favicons/favicon-192x192.png';
}
public function getName(){
switch($this->queriedContext) {
case 'Channel':
if(!empty($this->getInput('channel'))) {
return $this->getInput('channel');
}
break;
}
return parent::getName();
}
}

View File

@@ -9,7 +9,8 @@ class FurAffinityBridge extends BridgeAbstract {
'Search' => array(
'q' => array(
'name' => 'Query',
'required' => true
'required' => true,
'exampleValue' => 'dog',
),
'rating-general' => array(
'name' => 'General',
@@ -79,6 +80,7 @@ class FurAffinityBridge extends BridgeAbstract {
'limit' => array(
'name' => 'Limit',
'type' => 'number',
'required' => true,
'defaultValue' => 10,
'title' => 'Limit number of submissions to return. -1 for unlimited.'
),
@@ -449,6 +451,7 @@ class FurAffinityBridge extends BridgeAbstract {
'username-journals' => array(
'name' => 'Username',
'required' => true,
'exampleValue' => 'dhw',
'title' => 'Lowercase username as seen in URLs'
),
'limit' => array(
@@ -463,6 +466,7 @@ class FurAffinityBridge extends BridgeAbstract {
'journal-id' => array(
'name' => 'Journal ID',
'required' => true,
'exampleValue' => '10008853',
'type' => 'number',
'title' => 'Number seen in journal URL'
)
@@ -471,11 +475,13 @@ class FurAffinityBridge extends BridgeAbstract {
'username-gallery' => array(
'name' => 'Username',
'required' => true,
'exampleValue' => 'dhw',
'title' => 'Lowercase username as seen in URLs'
),
'limit' => array(
'name' => 'Limit',
'type' => 'number',
'required' => true,
'defaultValue' => 10,
'title' => 'Limit number of submissions to return. -1 for unlimited.'
),
@@ -496,11 +502,13 @@ class FurAffinityBridge extends BridgeAbstract {
'username-scraps' => array(
'name' => 'Username',
'required' => true,
'exampleValue' => 'dhw',
'title' => 'Lowercase username as seen in URLs'
),
'limit' => array(
'name' => 'Limit',
'type' => 'number',
'required' => true,
'defaultValue' => 10,
'title' => 'Limit number of submissions to return. -1 for unlimited.'
),
@@ -521,11 +529,13 @@ class FurAffinityBridge extends BridgeAbstract {
'username-favorites' => array(
'name' => 'Username',
'required' => true,
'exampleValue' => 'dhw',
'title' => 'Lowercase username as seen in URLs'
),
'limit' => array(
'name' => 'Limit',
'type' => 'number',
'required' => true,
'defaultValue' => 10,
'title' => 'Limit number of submissions to return. -1 for unlimited.'
),
@@ -546,17 +556,20 @@ class FurAffinityBridge extends BridgeAbstract {
'username-folder' => array(
'name' => 'Username',
'required' => true,
'exampleValue' => 'kopk',
'title' => 'Lowercase username as seen in URLs'
),
'folder-id' => array(
'name' => 'Folder ID',
'required' => true,
'exampleValue' => '1031990',
'type' => 'number',
'title' => 'Number seen in folder URL'
),
'limit' => array(
'name' => 'Limit',
'type' => 'number',
'required' => true,
'defaultValue' => 10,
'title' => 'Limit number of submissions to return. -1 for unlimited.'
),

View File

@@ -3,21 +3,23 @@ class FurAffinityUserBridge extends BridgeAbstract {
const NAME = 'FurAffinity User Gallery';
const URI = 'https://www.furaffinity.net';
const MAINTAINER = 'CyberJacob';
const DESCRIPTION = 'See https://rss-bridge.github.io/rss-bridge/Bridge_Specific/Furaffinityuser.html for explanation';
const PARAMETERS = array(
array(
'searchUsername' => array(
'name' => 'Search Username',
'type' => 'text',
'required' => true,
'title' => 'Username to fetch the gallery for'
'title' => 'Username to fetch the gallery for',
'exampleValue' => 'armundy',
),
'loginUsername' => array(
'name' => 'Login Username',
'aCookie' => array(
'name' => 'Login cookie \'a\'',
'type' => 'text',
'required' => true
),
'loginPassword' => array(
'name' => 'Login Password',
'bCookie' => array(
'name' => 'Login cookie \'b\'',
'type' => 'text',
'required' => true
)
@@ -25,10 +27,12 @@ class FurAffinityUserBridge extends BridgeAbstract {
);
public function collectData() {
$cookies = self::login();
$opt = array(CURLOPT_COOKIE => 'b=' . $this->getInput('bCookie') . '; a=' . $this->getInput('aCookie'));
$url = self::URI . '/gallery/' . $this->getInput('searchUsername');
$html = getSimpleHTMLDOM($url, $cookies);
$html = getSimpleHTMLDOM($url, array(), $opt)
or returnServerError('Could not load the user\'s gallery page.');
$submissions = $html->find('section[id=gallery-gallery]', 0)->find('figure');
foreach($submissions as $submission) {
@@ -51,59 +55,4 @@ class FurAffinityUserBridge extends BridgeAbstract {
public function getURI() {
return self::URI . '/user/' . $this->getInput('searchUsername');
}
private function login() {
$ch = curl_init(self::URI . '/login/');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_USERAGENT, ini_get('user_agent'));
curl_setopt($ch, CURLOPT_ENCODING, '');
curl_setopt($ch, CURLOPT_PROTOCOLS, CURLPROTO_HTTP | CURLPROTO_HTTPS);
$fields = implode('&', array(
'action=login',
'retard_protection=1',
'name=' . urlencode($this->getInput('loginUsername')),
'pass=' . urlencode($this->getInput('loginPassword')),
'login=Login to Faraffinity'
));
curl_setopt($ch, CURLOPT_POST, 5);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
if(defined('PROXY_URL') && !defined('NOPROXY')) {
curl_setopt($ch, CURLOPT_PROXY, PROXY_URL);
}
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
$data = curl_exec($ch);
$errorCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
$curlError = curl_error($ch);
$curlErrno = curl_errno($ch);
$curlInfo = curl_getinfo($ch);
if($data === false)
fDebug::log("Cant't download {$url} cUrl error: {$curlError} ({$curlErrno})");
curl_close($ch);
if($errorCode != 200) {
returnServerError(error_get_last());
} else {
preg_match_all('/^Set-Cookie:\s*([^;]*)/mi', $data, $matches);
$cookies = array();
foreach($matches[1] as $item) {
parse_str($item, $cookie);
$cookies = array_merge($cookies, $cookie);
}
return $cookies;
}
}
}

View File

@@ -32,45 +32,48 @@ class GBAtempBridge extends BridgeAbstract {
return $item;
}
private function decodeHtmlEntities($text) {
$text = html_entity_decode($text);
$convmap = array(0x0, 0x2FFFF, 0, 0xFFFF);
return trim(mb_decode_numericentity($text, $convmap, 'UTF-8'));
}
private function cleanupPostContent($content, $site_url){
$content = str_replace(':arrow:', '&#x27a4;', $content);
$content = str_replace('href="attachments/', 'href="' . $site_url . 'attachments/', $content);
$content = defaultLinkTo($content, self::URI);
$content = stripWithDelimiters($content, '<script', '</script>');
return $content;
$content = stripWithDelimiters($content, '<svg', '</svg>');
$content = stripRecursiveHTMLSection($content, 'div', '<div class="reactionsBar');
return $this->decodeHtmlEntities($content);
}
private function findItemDate($item){
$time = 0;
$dateField = $item->find('abbr.DateTime', 0);
$dateField = $item->find('time', 0);
if (is_object($dateField)) {
$time = intval(
extractFromDelimiters(
$dateField->outertext,
'data-time="',
'"'
)
);
} else {
$dateField = $item->find('span.DateTime', 0);
$time = DateTime::createFromFormat(
'M j, Y \a\t g:i A',
extractFromDelimiters(
$dateField->outertext,
'title="',
'"'
)
)->getTimestamp();
$time = strtotime($dateField->datetime);
}
return $time;
}
private function findItemImage($item, $selector){
$img = extractFromDelimiters($item->find($selector, 0)->style, 'url(', ')');
$paramPos = strpos($img, '?');
if ($paramPos !== false) {
$img = substr($img, 0, $paramPos);
}
if (!str_ends_with($img, '.png') && !str_ends_with($img, '.jpg')) {
$img = $img . '#.image';
}
return urljoin(self::URI, $img);
}
private function fetchPostContent($uri, $site_url){
$html = getSimpleHTMLDOMCached($uri);
if(!$html) {
return 'Could not request GBAtemp: ' . $uri;
}
$content = $html->find('div.messageContent, blockquote.baseHtml', 0)->innertext;
$content = $html->find('article.message-body', 0)->innertext;
return $this->cleanupPostContent($content, $site_url);
}
@@ -80,12 +83,12 @@ class GBAtempBridge extends BridgeAbstract {
switch($this->getInput('type')) {
case 'N':
foreach($html->find('li[class=news_item news full]') as $newsItem) {
$url = self::URI . $newsItem->find('a', 0)->href;
$img = $this->getURI() . extractFromDelimiters($newsItem->find('a.news_image', 0)->style, 'url(', ')') . '#.image';
foreach($html->find('li.news_item.full') as $newsItem) {
$url = urljoin(self::URI, $newsItem->find('a', 0)->href);
$img = $this->findItemImage($newsItem, 'a.news_image');
$time = $this->findItemDate($newsItem);
$author = $newsItem->find('a.username', 0)->plaintext;
$title = $newsItem->find('a', 1)->plaintext;
$title = $this->decodeHtmlEntities($newsItem->find('h3.news_title', 0)->plaintext);
$content = $this->fetchPostContent($url, self::URI);
$this->items[] = $this->buildItem($url, $title, $author, $time, $img, $content);
unset($newsItem); // Some items are heavy, freeing the item proactively helps saving memory
@@ -93,26 +96,23 @@ class GBAtempBridge extends BridgeAbstract {
break;
case 'R':
foreach($html->find('li.portal_review') as $reviewItem) {
$url = self::URI . $reviewItem->find('a', 0)->href;
$img = $this->getURI() . extractFromDelimiters($reviewItem->find('a', 0)->style, 'image:url(', ')');
$title = $reviewItem->find('span.review_title', 0)->plaintext;
$content = getSimpleHTMLDOM($url);
$author = $content->find('a.username', 0)->plaintext;
$url = urljoin(self::URI, $reviewItem->find('a.review_boxart', 0)->href);
$img = $this->findItemImage($reviewItem, 'a.review_boxart');
$title = $this->decodeHtmlEntities($reviewItem->find('h2.review_title', 0)->plaintext);
$content = getSimpleHTMLDOMCached($url);
$author = $content->find('span.author--name', 0)->plaintext;
$time = $this->findItemDate($content);
$intro = '<p><b>' . ($content->find('div#review_intro', 0)->plaintext) . '</b></p>';
$intro = '<p><b>' . ($content->find('div#review_introduction', 0)->plaintext) . '</b></p>';
$review = $content->find('div#review_main', 0)->innertext;
$subheader = '<p><b>' . $content->find('div.review_subheader', 0)->plaintext . '</b></p>';
$procons = $content->find('table.review_procons', 0)->outertext;
$scores = $content->find('table.reviewscores', 0)->outertext;
$content = $this->cleanupPostContent($intro . $review . $subheader . $procons . $scores, self::URI);
$content = $this->cleanupPostContent($intro . $review, self::URI);
$this->items[] = $this->buildItem($url, $title, $author, $time, $img, $content);
unset($reviewItem); // Free up memory
}
break;
case 'T':
foreach($html->find('li.portal-tutorial') as $tutorialItem) {
$url = self::URI . $tutorialItem->find('a', 1)->href;
$title = $tutorialItem->find('a', 1)->plaintext;
$url = urljoin(self::URI, $tutorialItem->find('a', 1)->href);
$title = $this->decodeHtmlEntities($tutorialItem->find('a', 1)->plaintext);
$time = $this->findItemDate($tutorialItem);
$author = $tutorialItem->find('a.username', 0)->plaintext;
$content = $this->fetchPostContent($url, self::URI);
@@ -122,8 +122,8 @@ class GBAtempBridge extends BridgeAbstract {
break;
case 'F':
foreach($html->find('li.rc_item') as $postItem) {
$url = self::URI . $postItem->find('a', 1)->href;
$title = $postItem->find('a', 1)->plaintext;
$url = urljoin(self::URI, $postItem->find('a', 1)->href);
$title = $this->decodeHtmlEntities($postItem->find('a', 1)->plaintext);
$time = $this->findItemDate($postItem);
$author = $postItem->find('a.username', 0)->plaintext;
$content = $this->fetchPostContent($url, self::URI);

View File

@@ -77,8 +77,6 @@ class GQMagazineBridge extends BridgeAbstract
// Since GQ don't want simple class scrapping, let's do it the hard way and ... discover content !
$main = $html->find('main', 0);
foreach ($main->find('a') as $link) {
if(strpos($link, $this->getInput('page')))
continue;
$uri = $link->href;
$date = $link->parent()->find('time', 0);
@@ -117,7 +115,7 @@ class GQMagazineBridge extends BridgeAbstract
*/
private function loadFullArticle($uri){
$html = getSimpleHTMLDOMCached($uri);
return $html->find('section[data-test-id=MainContentWrapper]', 0);
return $html->find('article', 0);
}
/**

View File

@@ -0,0 +1,54 @@
<?php
class GatesNotesBridge extends FeedExpander {
const MAINTAINER = 'corenting';
const NAME = 'Gates Notes';
const URI = 'https://www.gatesnotes.com';
const DESCRIPTION = 'Returns the newest articles.';
const CACHE_TIMEOUT = 21600; // 6h
protected function parseItem($item){
$item = parent::parseItem($item);
$article_html = getSimpleHTMLDOMCached($item['uri']);
if(!$article_html) {
$item['content'] .= '<p><em>Could not request ' . $this->getName() . ': ' . $item['uri'] . '</em></p>';
return $item;
}
$article_html = defaultLinkTo($article_html, $this->getURI());
$top_description = '<p>' . $article_html->find('div.article_top_description', 0)->innertext . '</p>';
$hero_image = '<img src=' . $article_html->find('img.article_top_DMT_Image', 0)->getAttribute('data-src') . '>';
$article_body = $article_html->find('div.TGN_Article_ReadTimeSection', 0);
// Convert iframe of Youtube videos to link
foreach($article_body->find('iframe') as $found) {
$iframeUrl = $found->getAttribute('src');
if ($iframeUrl) {
$text = 'Embedded Youtube video, click here to watch on Youtube.com';
$found->outertext = '<p><a href="' . $iframeUrl . '">' . $text . '</a></p>';
}
}
// Remove <link> CSS ressources
foreach($article_body->find('link') as $found) {
$linkedRessourceUrl = $found->getAttribute('href');
if (str_ends_with($linkedRessourceUrl, '.css')) {
$found->outertext = '';
}
}
$article_body = sanitize($article_body->innertext);
$item['content'] = $top_description . $hero_image . $article_body;
return $item;
}
public function collectData(){
$feed = static::URI . '/rss';
$this->collectExpandableDatas($feed);
}
}

View File

@@ -1,35 +1,87 @@
<?php
require_once('DanbooruBridge.php');
class GelbooruBridge extends DanbooruBridge {
class GelbooruBridge extends BridgeAbstract {
const MAINTAINER = 'mitsukarenai';
const NAME = 'Gelbooru';
const URI = 'http://gelbooru.com/';
const URI = 'https://gelbooru.com/';
const DESCRIPTION = 'Returns images from given page';
const PATHTODATA = '.thumb';
const IDATTRIBUTE = 'id';
const TAGATTRIBUTE = 'title';
const PIDBYPAGE = 63;
const PARAMETERS = array(
'global' => array(
'p' => array(
'name' => 'page',
'defaultValue' => 0,
'type' => 'number'
),
't' => array(
'name' => 'tags',
'exampleValue' => 'pinup',
'title' => 'Tags to search for'
),
'l' => array(
'name' => 'limit',
'exampleValue' => 100,
'title' => 'How many posts to retrieve (hard limit of 1000)'
)
),
0 => array()
);
protected function getFullURI(){
return $this->getURI()
. 'index.php?page=post&s=list&pid='
. ($this->getInput('p') ? ($this->getInput('p') - 1) * static::PIDBYPAGE : '')
. 'index.php?&page=dapi&s=post&q=index&json=1&pid=' . $this->getInput('p')
. '&limit=' . $this->getInput('l')
. '&tags=' . urlencode($this->getInput('t'));
}
protected function getTags($element){
$tags = parent::getTags($element);
$tags = explode(' ', $tags);
/*
This function is superfluous for GelbooruBridge, but useful
for Bridges that inherit from it
*/
protected function buildThumbnailURI($element){
return $this->getURI() . 'thumbnails/' . $element->directory
. '/thumbnail_' . $element->md5 . '.jpg';
}
// Remove statistics from the tags list (identified by colon)
foreach($tags as $key => $tag) {
if(strpos($tag, ':') !== false) unset($tags[$key]);
protected function getItemFromElement($element){
$item = array();
$item['uri'] = $this->getURI() . 'index.php?page=post&s=view&id='
. $element->id;
$item['postid'] = $element->id;
$item['author'] = $element->owner;
$item['timestamp'] = date('d F Y H:i:s', $element->change);
$item['tags'] = $element->tags;
$item['title'] = $this->getName() . ' | ' . $item['postid'];
if (isset($element->preview_url)) {
$thumbnailUri = $element->preview_url;
} else{
$thumbnailUri = $this->buildThumbnailURI($element);
}
return implode(' ', $tags);
$item['content'] = '<a href="' . $item['uri'] . '"><img src="'
. $thumbnailUri . '" /></a><br><br><b>Tags:</b> '
. $item['tags'] . '<br><br>' . $item['timestamp'];
return $item;
}
public function collectData(){
$content = getContents($this->getFullURI());
// Most other Gelbooru-based boorus put their content in the root of
// the JSON. This check is here for Bridges that inherit from this one
$posts = json_decode($content);
if (isset($posts->post)) {
$posts = $posts->post;
}
if (is_null($posts)) {
returnServerError('No posts found.');
}
foreach($posts as $post) {
$this->items[] = $this->getItemFromElement($post);
}
}
}

135
bridges/GettrBridge.php Normal file
View File

@@ -0,0 +1,135 @@
<?php
class GettrBridge extends BridgeAbstract
{
const NAME = 'Gettr.com bridge';
const URI = 'https://gettr.com';
const DESCRIPTION = 'Fetches the latest posts from a GETTR user';
const MAINTAINER = 'dvikan';
const CACHE_TIMEOUT = 60 * 15; // 15m
const PARAMETERS = [
[
'user' => [
'name' => 'User',
'type' => 'text',
'required' => true,
'exampleValue' => 'joerogan',
],
'limit' => [
'name' => 'Limit',
'type' => 'number',
'title' => 'Maximum number of items to return (maximum 20)',
'defaultValue' => 5,
'required' => true,
],
]
];
public function collectData()
{
$api = sprintf(
'https://api.gettr.com/u/user/%s/posts?offset=0&max=%s&dir=fwd&incl=posts&fp=f_uo',
$this->getInput('user'),
max($this->getInput('limit'), 20)
);
$data = json_decode(getContents($api), false);
foreach ($data->result->aux->post as $post) {
$this->items[] = [
'title' => mb_substr($post->txt ?? $post->uid . '@gettr.com', 0, 100),
'uri' => sprintf('https://gettr.com/post/%s', $post->_id),
'author' => $post->uid,
// Convert from ms to s
'timestamp' => substr($post->cdate, 0, strlen($post->cdate) - 3),
'uid' => $post->_id,
// Hashtags found within post text
'categories' => $post->htgs ?? [],
'content' => $this->createContent($post),
];
}
}
/**
* Collect text, image and video, if they exist
*/
private function createContent(\stdClass $post): string
{
$content = '';
// Text
if (isset($post->txt)) {
$isRepost = $this->getInput('user') !== $post->uid;
if ($isRepost) {
$content .= 'Reposted by ' . $this->getInput('user') . '@gettr.com<br><br>';
}
$content .= "$post->txt <br><br>";
}
// Preview image
if (isset($post->previmg)) {
$content .= <<<HTML
<a href="$post->prevsrc" target="_blank">
<img
src='$post->previmg'
alt='Unable to load image'
loading='lazy'
>
</a>
<br><br>
HTML;
}
// Images
foreach ($post->imgs ?? [] as $imageUrl) {
$content .= <<<HTML
<img
src='https://media.gettr.com/$imageUrl'
alt='Unable to load image'
target='_blank'
>
<br><br>
HTML;
}
// Video
if (isset($post->ovid)) {
$mainImage = $post->main;
$content .= <<<HTML
<video
style="max-width: 100%"
controls
preload="none"
poster="https://media.gettr.com/$mainImage"
>
<source src="https://media.gettr.com/$post->ovid" type="video/mp4">
Your browser does not support the video element. Kindly update it to latest version.
</video >
HTML;
// This is typically a m3u8 which I don't know how to present in a browser
$streamingUrl = $post->vid;
}
$this->processMetadata($post);
return $content;
}
public function getIcon()
{
return 'https://gettr.com/favicon.ico';
}
/**
* @param stdClass $post
*/
private function processMetadata(stdClass $post): void
{
// Unused metadata, maybe used later
$textLanguage = $post->txt_lang ?? 'en';
$replies = $post->cm ?? 0;
$likes = $post->lkbpst ?? 0;
$reposts = $post->shbpst ?? 0;
// I think a visibility of "p" means that it's public
$visibility = $post->vis ?? 'p';
}
}

65
bridges/GiphyBridge.php Executable file → Normal file
View File

@@ -11,8 +11,19 @@ class GiphyBridge extends BridgeAbstract {
const PARAMETERS = array( array(
's' => array(
'name' => 'search tag',
'exampleValue' => 'bird',
'required' => true
),
'noGif' => array(
'name' => 'Without gifs',
'type' => 'checkbox',
'title' => 'Exclude gifs from the results'
),
'noStick' => array(
'name' => 'Without stickers',
'type' => 'checkbox',
'title' => 'Exclude stickers from the results'
),
'n' => array(
'name' => 'max number of returned items (max 50)',
'type' => 'number',
@@ -20,25 +31,8 @@ class GiphyBridge extends BridgeAbstract {
)
));
public function collectData() {
/**
* This uses a public beta key which has severe rate limiting.
*
* https://giphy.api-docs.io/1.0/welcome/access-and-api-keys
* https://giphy.api-docs.io/1.0/gifs/search-1
*/
$apiKey = 'dc6zaTOxFJmzC';
$limit = min($this->getInput('n') ?: 10, 50);
$uri = sprintf(
'https://api.giphy.com/v1/gifs/search?q=%s&limit=%s&api_key=%s',
rawurlencode($this->getInput('s')),
$limit,
$apiKey
);
$result = json_decode(getContents($uri));
foreach($result->data as $entry) {
protected function getGiphyItems($entries){
foreach($entries as $entry) {
$createdAt = new \DateTime($entry->import_datetime);
$this->items[] = array(
@@ -50,6 +44,7 @@ class GiphyBridge extends BridgeAbstract {
'content' => <<<HTML
<a href="{$entry->url}">
<img
loading="lazy"
src="{$entry->images->downsized->url}"
width="{$entry->images->downsized->width}"
height="{$entry->images->downsized->height}" />
@@ -57,6 +52,38 @@ class GiphyBridge extends BridgeAbstract {
HTML
);
}
}
public function collectData() {
/**
* This uses a public beta key which has severe rate limiting.
*
* https://giphy.api-docs.io/1.0/welcome/access-and-api-keys
* https://giphy.api-docs.io/1.0/gifs/search-1
*/
$apiKey = 'dc6zaTOxFJmzC';
$limit = min($this->getInput('n') ?: 10, 50);
$endpoints = array();
if (empty($this->getInput('noGif'))) {
$endpoints[] = 'gifs';
}
if (empty($this->getInput('noStick'))) {
$endpoints[] = 'stickers';
}
foreach ($endpoints as $endpoint) {
$uri = sprintf(
'https://api.giphy.com/v1/%s/search?q=%s&limit=%s&api_key=%s',
$endpoint,
rawurlencode($this->getInput('s')),
$limit,
$apiKey
);
$result = json_decode(getContents($uri));
$this->getGiphyItems($result->data);
}
usort($this->items, function ($a, $b) {
return $a['timestamp'] < $b['timestamp'];

View File

@@ -14,7 +14,7 @@ class GitHubGistBridge extends BridgeAbstract {
'type' => 'text',
'required' => true,
'title' => 'Insert Gist ID or URI',
'exampleValue' => '2646763, https://gist.github.com/2646763'
'exampleValue' => '2646763'
)
));
@@ -71,8 +71,7 @@ class GitHubGistBridge extends BridgeAbstract {
$uri = $comment->find('a[href*=#gistcomment]', 0)
or returnServerError('Could not find comment anchor!');
$title = $comment->find('div[class~="unminimized-comment"] h3[class~="timeline-comment-header-text"]', 0)
or returnServerError('Could not find comment header text!');
$title = $comment->find('h3', 0);
$datetime = $comment->find('[datetime]', 0)
or returnServerError('Could not find comment datetime!');
@@ -86,7 +85,7 @@ class GitHubGistBridge extends BridgeAbstract {
$item = array();
$item['uri'] = $uri->href;
$item['title'] = str_replace('commented', 'commented on', $title->plaintext);
$item['title'] = str_replace('commented', 'commented on', $title->plaintext ?? '');
$item['timestamp'] = strtotime($datetime->datetime);
$item['author'] = '<a href="' . $author->href . '">' . $author->plaintext . '</a>';
$item['content'] = $this->fixContent($message);

View File

@@ -4,17 +4,19 @@ class GithubIssueBridge extends BridgeAbstract {
const MAINTAINER = 'Pierre Mazière';
const NAME = 'Github Issue';
const URI = 'https://github.com/';
const CACHE_TIMEOUT = 600; // 10min
const CACHE_TIMEOUT = 0; // 10min
const DESCRIPTION = 'Returns the issues or comments of an issue of a github project';
const PARAMETERS = array(
'global' => array(
'u' => array(
'name' => 'User name',
'exampleValue' => 'RSS-Bridge',
'required' => true
),
'p' => array(
'name' => 'Project name',
'exampleValue' => 'rss-bridge',
'required' => true
)
),
@@ -22,12 +24,18 @@ class GithubIssueBridge extends BridgeAbstract {
'c' => array(
'name' => 'Show Issues Comments',
'type' => 'checkbox'
),
'q' => array(
'name' => 'Search Query',
'defaultValue' => 'is:issue is:open sort:updated-desc',
'required' => true
)
),
'Issue comments' => array(
'i' => array(
'name' => 'Issue number',
'type' => 'number',
'exampleValue' => '2099',
'required' => true
)
)
@@ -37,7 +45,6 @@ class GithubIssueBridge extends BridgeAbstract {
const BRIDGE_OPTIONS = array(0 => 'Project Issues', 1 => 'Issue comments');
const URL_PATH = 'issues';
const SEARCH_QUERY_PATH = 'issues';
const SEARCH_QUERY = '?q=is%3Aissue+sort%3Aupdated-desc';
public function getName(){
$name = $this->getInput('u') . '/' . $this->getInput('p');
@@ -64,7 +71,7 @@ class GithubIssueBridge extends BridgeAbstract {
if($this->queriedContext === static::BRIDGE_OPTIONS[1]) {
$uri .= static::URL_PATH . '/' . $this->getInput('i');
} else {
$uri .= static::SEARCH_QUERY_PATH . static::SEARCH_QUERY;
$uri .= static::SEARCH_QUERY_PATH . '?q=' . urlencode($this->getInput('q'));
}
return $uri;
}
@@ -125,9 +132,8 @@ class GithubIssueBridge extends BridgeAbstract {
$author = $comment->find('.author', 0)->plaintext;
$title .= ' / ' . trim(
$comment->find('.timeline-comment-header-text', 0)->plaintext
);
$header = $comment->find('.timeline-comment-header > h3', 0);
$title .= ' / ' . ($header ? $header->plaintext : 'Activity');
$time = $comment->find('relative-time', 0);
if ($time === null) {

View File

@@ -9,10 +9,12 @@ class GitHubPullRequestBridge extends GithubIssueBridge {
'global' => array(
'u' => array(
'name' => 'User name',
'exampleValue' => 'RSS-Bridge',
'required' => true
),
'p' => array(
'name' => 'Project name',
'exampleValue' => 'rss-bridge',
'required' => true
)
),
@@ -20,12 +22,18 @@ class GitHubPullRequestBridge extends GithubIssueBridge {
'c' => array(
'name' => 'Show Pull Request Comments',
'type' => 'checkbox'
),
'q' => array(
'name' => 'Search Query',
'defaultValue' => 'is:pr is:open sort:created-desc',
'required' => true
)
),
'Pull Request comments' => array(
'i' => array(
'name' => 'Pull Request number',
'type' => 'number',
'exampleValue' => '2100',
'required' => true
)
)
@@ -34,5 +42,4 @@ class GitHubPullRequestBridge extends GithubIssueBridge {
const BRIDGE_OPTIONS = array(0 => 'Project Pull Requests', 1 => 'Pull Request comments');
const URL_PATH = 'pull';
const SEARCH_QUERY_PATH = 'pulls';
const SEARCH_QUERY = '?q=is%3Apr+sort%3Aupdated-desc';
}

View File

@@ -9,6 +9,8 @@ class GithubSearchBridge extends BridgeAbstract {
const PARAMETERS = array( array(
's' => array(
'type' => 'text',
'required' => true,
'exampleValue' => 'rss-bridge',
'name' => 'Search query'
)
));

View File

@@ -8,41 +8,39 @@ class GithubTrendingBridge extends BridgeAbstract {
const CACHE_TIMEOUT = 43200; // 12hr
const DESCRIPTION = 'See what the GitHub community is most excited repos.';
const PARAMETERS = array(
// If you are changing context and/or parameter names, change them also in getName().
'By language' => array(
'language' => array(
'name' => 'Select language',
'type' => 'list',
'values' => array(
'All languages' => '',
'C++' => 'c++',
'HTML' => 'html',
'Java' => 'java',
'JavaScript' => 'javascript',
'PHP' => 'php',
'Python' => 'python',
'Ruby' => 'ruby',
'Unknown languages' => 'unknown languages',
'1C Enterprise' => '1c enterprise',
'Shell' => 'shell',
'Unknown languages' => 'unknown',
'1C Enterprise' => '1c-enterprise',
'4D' => '4d',
'ABAP' => 'abap',
'ABNF' => 'abnf',
'ActionScript' => 'actionscript',
'Ada' => 'ada',
'Adobe Font Metrics' => 'adobe font metrics',
'Adobe Font Metrics' => 'adobe-font-metrics',
'Agda' => 'agda',
'AGS Script' => 'ags script',
'AGS Script' => 'ags-script',
'Alloy' => 'alloy',
'Alpine Abuild' => 'alpine abuild',
'Altium Designer' => 'altium designer',
'Alpine Abuild' => 'alpine-abuild',
'Altium Designer' => 'altium-designer',
'AMPL' => 'ampl',
'AngelScript' => 'angelscript',
'Ant Build System' => 'ant build system',
'Ant Build System' => 'ant-build-system',
'ANTLR' => 'antlr',
'ApacheConf' => 'apacheconf',
'Apex' => 'apex',
'API Blueprint' => 'api blueprint',
'API Blueprint' => 'api-blueprint',
'APL' => 'apl',
'Apollo Guidance Computer' => 'apollo guidance computer',
'Apollo Guidance Computer' => 'apollo-guidance-computer',
'AppleScript' => 'applescript',
'Arc' => 'arc',
'AsciiDoc' => 'asciidoc',
@@ -71,11 +69,12 @@ class GithubTrendingBridge extends BridgeAbstract {
'Brightscript' => 'brightscript',
'Zeek' => 'zeek',
'C' => 'c',
'C#' => 'c#',
'C#' => 'c%23', // already URL encoded
'C++' => 'c++',
'C-ObjDump' => 'c-objdump',
'C2hs Haskell' => 'c2hs haskell',
'Cabal Config' => 'cabal config',
'C2hs Haskell' => 'c2hs-haskell',
'Cabal Config' => 'cabal-config',
'Cap\'n Proto' => 'cap\'n-proto',
'CartoCSS' => 'cartocss',
'Ceylon' => 'ceylon',
'Chapel' => 'chapel',
@@ -87,18 +86,18 @@ class GithubTrendingBridge extends BridgeAbstract {
'Click' => 'click',
'CLIPS' => 'clips',
'Clojure' => 'clojure',
'Closure Templates' => 'closure templates',
'Cloud Firestore Security Rules' => 'cloud firestore security rules',
'Closure Templates' => 'closure-templates',
'Cloud Firestore Security Rules' => 'cloud-firestore-security-rules',
'CMake' => 'cmake',
'COBOL' => 'cobol',
'CodeQL' => 'codeql',
'CoffeeScript' => 'coffeescript',
'ColdFusion' => 'coldfusion',
'ColdFusion CFC' => 'coldfusion cfc',
'ColdFusion CFC' => 'coldfusion-cfc',
'COLLADA' => 'collada',
'Common Lisp' => 'common lisp',
'Common Workflow Language' => 'common workflow language',
'Component Pascal' => 'component pascal',
'Common Lisp' => 'common-lisp',
'Common Workflow Language' => 'common-workflow-language',
'Component Pascal' => 'component-pascal',
'CoNLL-U' => 'conll-u',
'Cool' => 'cool',
'Coq' => 'coq',
@@ -107,28 +106,28 @@ class GithubTrendingBridge extends BridgeAbstract {
'Crystal' => 'crystal',
'CSON' => 'cson',
'Csound' => 'csound',
'Csound Document' => 'csound document',
'Csound Score' => 'csound score',
'Csound Document' => 'csound-document',
'Csound Score' => 'csound-score',
'CSS' => 'css',
'CSV' => 'csv',
'Cuda' => 'cuda',
'cURL Config' => 'curl config',
'cURL Config' => 'curl-config',
'CWeb' => 'cweb',
'Cycript' => 'cycript',
'Cython' => 'cython',
'D' => 'd',
'D-ObjDump' => 'd-objdump',
'Darcs Patch' => 'darcs patch',
'Darcs Patch' => 'darcs-patch',
'Dart' => 'dart',
'DataWeave' => 'dataweave',
'desktop' => 'desktop',
'Dhall' => 'dhall',
'Diff' => 'diff',
'DIGITAL Command Language' => 'digital command language',
'DIGITAL Command Language' => 'digital-command-language',
'dircolors' => 'dircolors',
'DirectX 3D File' => 'directx 3d file',
'DirectX 3D File' => 'directx-3d-file',
'DM' => 'dm',
'DNS Zone' => 'dns zone',
'DNS Zone' => 'dns-zone',
'Dockerfile' => 'dockerfile',
'Dogescript' => 'dogescript',
'DTrace' => 'dtrace',
@@ -138,29 +137,29 @@ class GithubTrendingBridge extends BridgeAbstract {
'Easybuild' => 'easybuild',
'EBNF' => 'ebnf',
'eC' => 'ec',
'Ecere Projects' => 'ecere projects',
'Ecere Projects' => 'ecere-projects',
'ECL' => 'ecl',
'ECLiPSe' => 'eclipse',
'EditorConfig' => 'editorconfig',
'Edje Data Collection' => 'edje data collection',
'Edje Data Collection' => 'edje-data-collection',
'edn' => 'edn',
'Eiffel' => 'eiffel',
'EJS' => 'ejs',
'Elixir' => 'elixir',
'Elm' => 'elm',
'Emacs Lisp' => 'emacs lisp',
'Emacs Lisp' => 'emacs-lisp',
'EmberScript' => 'emberscript',
'EML' => 'eml',
'EQ' => 'eq',
'Erlang' => 'erlang',
'F#' => 'f#',
'F#' => 'f%23', // already URL encoded
'F*' => 'f*',
'Factor' => 'factor',
'Fancy' => 'fancy',
'Fantom' => 'fantom',
'Faust' => 'faust',
'FIGlet Font' => 'figlet font',
'Filebench WML' => 'filebench wml',
'FIGlet Font' => 'figlet-font',
'Filebench WML' => 'filebench-wml',
'Filterscript' => 'filterscript',
'fish' => 'fish',
'FLUX' => 'flux',
@@ -170,25 +169,25 @@ class GithubTrendingBridge extends BridgeAbstract {
'FreeMarker' => 'freemarker',
'Frege' => 'frege',
'G-code' => 'g-code',
'Game Maker Language' => 'game maker language',
'Game Maker Language' => 'game-maker-language',
'GAML' => 'gaml',
'GAMS' => 'gams',
'GAP' => 'gap',
'GCC Machine Description' => 'gcc machine description',
'GCC Machine Description' => 'gcc-machine-description',
'GDB' => 'gdb',
'GDScript' => 'gdscript',
'Genie' => 'genie',
'Genshi' => 'genshi',
'Gentoo Ebuild' => 'gentoo ebuild',
'Gentoo Eclass' => 'gentoo eclass',
'Gerber Image' => 'gerber image',
'Gettext Catalog' => 'gettext catalog',
'Gentoo Ebuild' => 'gentoo-ebuild',
'Gentoo Eclass' => 'gentoo-eclass',
'Gerber Image' => 'gerber-image',
'Gettext Catalog' => 'gettext-catalog',
'Gherkin' => 'gherkin',
'Git Attributes' => 'git attributes',
'Git Config' => 'git config',
'Git Attributes' => 'git-attributes',
'Git Config' => 'git-config',
'GLSL' => 'glsl',
'Glyph' => 'glyph',
'Glyph Bitmap Distribution Format' => 'glyph bitmap distribution format',
'Glyph Bitmap Distribution Format' => 'glyph-bitmap-distribution-format',
'GN' => 'gn',
'Gnuplot' => 'gnuplot',
'Go' => 'go',
@@ -196,12 +195,12 @@ class GithubTrendingBridge extends BridgeAbstract {
'Gosu' => 'gosu',
'Grace' => 'grace',
'Gradle' => 'gradle',
'Grammatical Framework' => 'grammatical framework',
'Graph Modeling Language' => 'graph modeling language',
'Grammatical Framework' => 'grammatical-framework',
'Graph Modeling Language' => 'graph-modeling-language',
'GraphQL' => 'graphql',
'Graphviz (DOT)' => 'graphviz (dot)',
'Graphviz (DOT)' => 'graphviz-(dot)',
'Groovy' => 'groovy',
'Groovy Server Pages' => 'groovy server pages',
'Groovy Server Pages' => 'groovy-server-pages',
'Hack' => 'hack',
'Haml' => 'haml',
'Handlebars' => 'handlebars',
@@ -213,7 +212,6 @@ class GithubTrendingBridge extends BridgeAbstract {
'HiveQL' => 'hiveql',
'HLSL' => 'hlsl',
'HolyC' => 'holyc',
'HTML' => 'html',
'HTML+Django' => 'html+django',
'HTML+ECR' => 'html+ecr',
'HTML+EEX' => 'html+eex',
@@ -226,39 +224,39 @@ class GithubTrendingBridge extends BridgeAbstract {
'HyPhy' => 'hyphy',
'IDL' => 'idl',
'Idris' => 'idris',
'Ignore List' => 'ignore list',
'IGOR Pro' => 'igor pro',
'Inform 7' => 'inform 7',
'Ignore List' => 'ignore-list',
'IGOR Pro' => 'igor-pro',
'Inform 7' => 'inform-7',
'INI' => 'ini',
'Inno Setup' => 'inno setup',
'Inno Setup' => 'inno-setup',
'Io' => 'io',
'Ioke' => 'ioke',
'IRC log' => 'irc log',
'IRC log' => 'irc-log',
'Isabelle' => 'isabelle',
'Isabelle ROOT' => 'isabelle root',
'Isabelle ROOT' => 'isabelle-root',
'J' => 'j',
'Jasmin' => 'jasmin',
'Java' => 'java',
'Java Properties' => 'java properties',
'Java Server Pages' => 'java server pages',
'Java Properties' => 'java-properties',
'Java Server Pages' => 'java-server-pages',
'JavaScript' => 'javascript',
'JavaScript+ERB' => 'javascript+erb',
'JFlex' => 'jflex',
'Jison' => 'jison',
'Jison Lex' => 'jison lex',
'Jison Lex' => 'jison-lex',
'Jolie' => 'jolie',
'JSON' => 'json',
'JSON with Comments' => 'json with comments',
'JSON with Comments' => 'json-with-comments',
'JSON5' => 'json5',
'JSONiq' => 'jsoniq',
'JSONLD' => 'jsonld',
'Jsonnet' => 'jsonnet',
'JSX' => 'jsx',
'Julia' => 'julia',
'Jupyter Notebook' => 'jupyter notebook',
'KiCad Layout' => 'kicad layout',
'KiCad Legacy Layout' => 'kicad legacy layout',
'KiCad Schematic' => 'kicad schematic',
'Jupyter Notebook' => 'jupyter-notebook',
'KiCad Layout' => 'kicad-layout',
'KiCad Legacy Layout' => 'kicad-legacy-layout',
'KiCad Schematic' => 'kicad-schematic',
'Kit' => 'kit',
'Kotlin' => 'kotlin',
'KRL' => 'krl',
@@ -271,12 +269,12 @@ class GithubTrendingBridge extends BridgeAbstract {
'LFE' => 'lfe',
'LilyPond' => 'lilypond',
'Limbo' => 'limbo',
'Linker Script' => 'linker script',
'Linux Kernel Module' => 'linux kernel module',
'Linker Script' => 'linker-script',
'Linux Kernel Module' => 'linux-kernel-module',
'Liquid' => 'liquid',
'Literate Agda' => 'literate agda',
'Literate CoffeeScript' => 'literate coffeescript',
'Literate Haskell' => 'literate haskell',
'Literate Agda' => 'literate-agda',
'Literate CoffeeScript' => 'literate-coffeescript',
'Literate Haskell' => 'literate-haskell',
'LiveScript' => 'livescript',
'LLVM' => 'llvm',
'Logos' => 'logos',
@@ -285,7 +283,7 @@ class GithubTrendingBridge extends BridgeAbstract {
'LookML' => 'lookml',
'LoomScript' => 'loomscript',
'LSL' => 'lsl',
'LTspice Symbol' => 'ltspice symbol',
'LTspice Symbol' => 'ltspice-symbol',
'Lua' => 'lua',
'M' => 'm',
'M4' => 'm4',
@@ -297,7 +295,7 @@ class GithubTrendingBridge extends BridgeAbstract {
'Mask' => 'mask',
'Mathematica' => 'mathematica',
'MATLAB' => 'matlab',
'Maven POM' => 'maven pom',
'Maven POM' => 'maven-pom',
'Max' => 'max',
'MAXScript' => 'maxscript',
'mcfunction' => 'mcfunction',
@@ -305,19 +303,19 @@ class GithubTrendingBridge extends BridgeAbstract {
'Mercury' => 'mercury',
'Meson' => 'meson',
'Metal' => 'metal',
'Microsoft Developer Studio Project' => 'microsoft developer studio project',
'Microsoft Developer Studio Project' => 'microsoft-developer-studio-project',
'MiniD' => 'minid',
'Mirah' => 'mirah',
'mIRC Script' => 'mirc script',
'mIRC Script' => 'mirc-script',
'MLIR' => 'mlir',
'Modelica' => 'modelica',
'Modula-2' => 'modula-2',
'Modula-3' => 'modula-3',
'Module Management System' => 'module management system',
'Module Management System' => 'module-management-system',
'Monkey' => 'monkey',
'Moocode' => 'moocode',
'MoonScript' => 'moonscript',
'Motorola 68K Assembly' => 'motorola 68k assembly',
'Motorola 68K Assembly' => 'motorola-68k-assembly',
'MQL4' => 'mql4',
'MQL5' => 'mql5',
'MTML' => 'mtml',
@@ -342,12 +340,12 @@ class GithubTrendingBridge extends BridgeAbstract {
'Nit' => 'nit',
'Nix' => 'nix',
'NL' => 'nl',
'NPM Config' => 'npm config',
'NPM Config' => 'npm-config',
'NSIS' => 'nsis',
'Nu' => 'nu',
'NumPy' => 'numpy',
'ObjDump' => 'objdump',
'Object Data Instance Notation' => 'object data instance notation',
'Object Data Instance Notation' => 'object-data-instance-notation',
'Objective-C' => 'objective-c',
'Objective-C++' => 'objective-c++',
'Objective-J' => 'objective-j',
@@ -358,14 +356,14 @@ class GithubTrendingBridge extends BridgeAbstract {
'ooc' => 'ooc',
'Opa' => 'opa',
'Opal' => 'opal',
'Open Policy Agent' => 'open policy agent',
'Open Policy Agent' => 'open-policy-agent',
'OpenCL' => 'opencl',
'OpenEdge ABL' => 'openedge abl',
'OpenEdge ABL' => 'openedge-abl',
'OpenQASM' => 'openqasm',
'OpenRC runscript' => 'openrc runscript',
'OpenRC runscript' => 'openrc-runscript',
'OpenSCAD' => 'openscad',
'OpenStep Property List' => 'openstep property list',
'OpenType Feature File' => 'opentype feature file',
'OpenStep Property List' => 'openstep-property-list',
'OpenType Feature File' => 'opentype-feature-file',
'Org' => 'org',
'Ox' => 'ox',
'Oxygene' => 'oxygene',
@@ -374,13 +372,12 @@ class GithubTrendingBridge extends BridgeAbstract {
'Pan' => 'pan',
'Papyrus' => 'papyrus',
'Parrot' => 'parrot',
'Parrot Assembly' => 'parrot assembly',
'Parrot Internal Representation' => 'parrot internal representation',
'Parrot Assembly' => 'parrot-assembly',
'Parrot Internal Representation' => 'parrot-internal-representation',
'Pascal' => 'pascal',
'Pawn' => 'pawn',
'Pep8' => 'pep8',
'Perl' => 'perl',
'PHP' => 'php',
'Pic' => 'pic',
'Pickle' => 'pickle',
'PicoLisp' => 'picolisp',
@@ -389,29 +386,28 @@ class GithubTrendingBridge extends BridgeAbstract {
'PLpgSQL' => 'plpgsql',
'PLSQL' => 'plsql',
'Pod' => 'pod',
'Pod 6' => 'pod 6',
'Pod 6' => 'pod-6',
'PogoScript' => 'pogoscript',
'Pony' => 'pony',
'PostCSS' => 'postcss',
'PostScript' => 'postscript',
'POV-Ray SDL' => 'pov-ray sdl',
'POV-Ray SDL' => 'pov-ray-sdl',
'PowerBuilder' => 'powerbuilder',
'PowerShell' => 'powershell',
'Prisma' => 'prisma',
'Processing' => 'processing',
'Proguard' => 'proguard',
'Prolog' => 'prolog',
'Propeller Spin' => 'propeller spin',
'Protocol Buffer' => 'protocol buffer',
'Public Key' => 'public key',
'Propeller Spin' => 'propeller-spin',
'Protocol Buffer' => 'protocol-buffer',
'Public Key' => 'public-key',
'Pug' => 'pug',
'Puppet' => 'puppet',
'Pure Data' => 'pure data',
'Pure Data' => 'pure-data',
'PureBasic' => 'purebasic',
'PureScript' => 'purescript',
'Python' => 'python',
'Python console' => 'python console',
'Python traceback' => 'python traceback',
'Python console' => 'python-console',
'Python traceback' => 'python-traceback',
'q' => 'q',
'QMake' => 'qmake',
'QML' => 'qml',
@@ -422,30 +418,30 @@ class GithubTrendingBridge extends BridgeAbstract {
'Raku' => 'raku',
'RAML' => 'raml',
'Rascal' => 'rascal',
'Raw token data' => 'raw token data',
'Raw token data' => 'raw-token-data',
'RDoc' => 'rdoc',
'Readline Config' => 'readline config',
'Readline Config' => 'readline-config',
'REALbasic' => 'realbasic',
'Reason' => 'reason',
'Rebol' => 'rebol',
'Red' => 'red',
'Redcode' => 'redcode',
'Regular Expression' => 'regular expression',
// 'Ren'Py' => 'ren'py',
'Regular Expression' => 'regular-expression',
'Ren\'Py' => 'ren\'py',
'RenderScript' => 'renderscript',
'reStructuredText' => 'restructuredtext',
'REXX' => 'rexx',
'RHTML' => 'rhtml',
'Rich Text Format' => 'rich text format',
'Rich Text Format' => 'rich-text-format',
'Ring' => 'ring',
'Riot' => 'riot',
'RMarkdown' => 'rmarkdown',
'RobotFramework' => 'robotframework',
'Roff' => 'roff',
'Roff Manpage' => 'roff manpage',
'Roff Manpage' => 'roff-manpage',
'Rouge' => 'rouge',
'RPC' => 'rpc',
'RPM Spec' => 'rpm spec',
'RPM Spec' => 'rpm-spec',
'Ruby' => 'ruby',
'RUNOFF' => 'runoff',
'Rust' => 'rust',
@@ -461,7 +457,6 @@ class GithubTrendingBridge extends BridgeAbstract {
'sed' => 'sed',
'Self' => 'self',
'ShaderLab' => 'shaderlab',
'Shell' => 'shell',
'ShellSession' => 'shellsession',
'Shen' => 'shen',
'Slash' => 'slash',
@@ -475,20 +470,20 @@ class GithubTrendingBridge extends BridgeAbstract {
'Solidity' => 'solidity',
'SourcePawn' => 'sourcepawn',
'SPARQL' => 'sparql',
'Spline Font Database' => 'spline font database',
'Spline Font Database' => 'spline-font-database',
'SQF' => 'sqf',
'SQL' => 'sql',
'SQLPL' => 'sqlpl',
'Squirrel' => 'squirrel',
'SRecode Template' => 'srecode template',
'SSH Config' => 'ssh config',
'SRecode Template' => 'srecode-template',
'SSH Config' => 'ssh-config',
'Stan' => 'stan',
'Standard ML' => 'standard ml',
'Standard ML' => 'standard-ml',
'Starlark' => 'starlark',
'Stata' => 'stata',
'STON' => 'ston',
'Stylus' => 'stylus',
'SubRip Text' => 'subrip text',
'SubRip Text' => 'subrip-text',
'SugarSS' => 'sugarss',
'SuperCollider' => 'supercollider',
'Svelte' => 'svelte',
@@ -505,7 +500,7 @@ class GithubTrendingBridge extends BridgeAbstract {
'Text' => 'text',
'Textile' => 'textile',
'Thrift' => 'thrift',
'TI Program' => 'ti program',
'TI Program' => 'ti-program',
'TLA' => 'tla',
'TOML' => 'toml',
'TSQL' => 'tsql',
@@ -514,11 +509,11 @@ class GithubTrendingBridge extends BridgeAbstract {
'Turtle' => 'turtle',
'Twig' => 'twig',
'TXL' => 'txl',
'Type Language' => 'type language',
'Type Language' => 'type-language',
'TypeScript' => 'typescript',
'Unified Parallel C' => 'unified parallel c',
'Unity3D Asset' => 'unity3d asset',
'Unix Assembly' => 'unix assembly',
'Unified Parallel C' => 'unified-parallel-c',
'Unity3D Asset' => 'unity3d-asset',
'Unix Assembly' => 'unix-assembly',
'Uno' => 'uno',
'UnrealScript' => 'unrealscript',
'UrWeb' => 'urweb',
@@ -529,33 +524,33 @@ class GithubTrendingBridge extends BridgeAbstract {
'VCL' => 'vcl',
'Verilog' => 'verilog',
'VHDL' => 'vhdl',
'Vim script' => 'vim script',
'Vim Snippet' => 'vim snippet',
'Visual Basic .NET' => 'visual basic .net',
'Visual Basic .NET' => 'visual basic .net',
'Vim script' => 'vim-script',
'Vim Snippet' => 'vim-snippet',
'Visual Basic .NET' => 'visual-basic-.net',
'Visual Basic .NET' => 'visual-basic-.net',
'Volt' => 'volt',
'Vue' => 'vue',
'Wavefront Material' => 'wavefront material',
'Wavefront Object' => 'wavefront object',
'Wavefront Material' => 'wavefront-material',
'Wavefront Object' => 'wavefront-object',
'wdl' => 'wdl',
'Web Ontology Language' => 'web ontology language',
'Web Ontology Language' => 'web-ontology-language',
'WebAssembly' => 'webassembly',
'WebIDL' => 'webidl',
'WebVTT' => 'webvtt',
'Wget Config' => 'wget config',
'Windows Registry Entries' => 'windows registry entries',
'Wget Config' => 'wget-config',
'Windows Registry Entries' => 'windows-registry-entries',
'wisp' => 'wisp',
'Wollok' => 'wollok',
'World of Warcraft Addon Data' => 'world of warcraft addon data',
'X BitMap' => 'x bitmap',
'X Font Directory Index' => 'x font directory index',
'X PixMap' => 'x pixmap',
'World of Warcraft Addon Data' => 'world-of-warcraft-addon-data',
'X BitMap' => 'x-bitmap',
'X Font Directory Index' => 'x-font-directory-index',
'X PixMap' => 'x-pixmap',
'X10' => 'x10',
'xBase' => 'xbase',
'XC' => 'xc',
'XCompose' => 'xcompose',
'XML' => 'xml',
'XML Property List' => 'xml property list',
'XML Property List' => 'xml-property-list',
'Xojo' => 'xojo',
'XPages' => 'xpages',
'XProc' => 'xproc',
@@ -584,7 +579,6 @@ class GithubTrendingBridge extends BridgeAbstract {
'date_range' => array(
'name' => 'Date range',
'type' => 'list',
'required' => false,
'values' => array(
'Today' => 'today',
'Weekly' => 'weekly',
@@ -613,7 +607,9 @@ class GithubTrendingBridge extends BridgeAbstract {
$item['title'] = str_replace(' ', '', trim(strip_tags($element->find('h1 a', 0)->plaintext)));
// Description
$item['content'] = trim(strip_tags($element->find('p.text-gray', 0)->innertext));
$description = $element->find('p', 0);
if ($description != null)
$item['content'] = trim(strip_tags($description->innertext));
// Time
$item['timestamp'] = time();
@@ -624,10 +620,9 @@ class GithubTrendingBridge extends BridgeAbstract {
}
public function getName(){
if($this->getInput('language') == '') {
return self::NAME . ': all';
} elseif (!is_null($this->getInput('language'))) {
return self::NAME . ': ' . $this->getInput('language');
if (!is_null($this->getInput('language'))) {
$language = array_search($this->getInput('language'), self::PARAMETERS['By language']['language']['values']);
return self::NAME . ': ' . $language;
}
return parent::getName();

View File

@@ -1,4 +1,5 @@
<?php
class GlassdoorBridge extends BridgeAbstract {
// Contexts
@@ -17,7 +18,6 @@ class GlassdoorBridge extends BridgeAbstract {
const BLOG_TYPE_COMPANIES_HIRING = 'Companies Hiring';
const BLOG_TYPE_CAREER_ADVICE = 'Career Advice';
const BLOG_TYPE_INTERVIEWS = 'Interviews';
const BLOG_TYPE_GUIDE = 'Guides';
// Review context parameters
const PARAM_REVIEW_COMPANY = 'company';
@@ -39,7 +39,6 @@ class GlassdoorBridge extends BridgeAbstract {
self::BLOG_TYPE_COMPANIES_HIRING => 'blog/companies-hiring/',
self::BLOG_TYPE_CAREER_ADVICE => 'blog/career-advice/',
self::BLOG_TYPE_INTERVIEWS => 'blog/interviews/',
self::BLOG_TYPE_GUIDE => 'blog/guide/'
)
),
self::PARAM_BLOG_FULL => array(
@@ -67,9 +66,6 @@ class GlassdoorBridge extends BridgeAbstract {
)
);
private $host = self::URI; // They redirect without notice :/
private $title = '';
public function getURI() {
switch($this->queriedContext) {
case self::CONTEXT_BLOG:
@@ -81,18 +77,10 @@ class GlassdoorBridge extends BridgeAbstract {
return parent::getURI();
}
public function getName() {
return $this->title ? $this->title . ' - ' . self::NAME : parent::getName();
}
public function collectData() {
$html = getSimpleHTMLDOM($this->getURI());
$this->host = $html->find('link[rel="canonical"]', 0)->href;
$html = defaultLinkTo($html, $this->host);
$this->title = $html->find('meta[property="og:title"]', 0)->content;
$url = $this->getURI();
$html = getSimpleHTMLDOM($url);
$html = defaultLinkTo($html, $url);
$limit = $this->getInput(self::PARAM_LIMIT);
switch($this->queriedContext) {
@@ -106,35 +94,24 @@ class GlassdoorBridge extends BridgeAbstract {
}
private function collectBlogData($html, $limit) {
$posts = $html->find('section')
$posts = $html->find('div.post')
or returnServerError('Unable to find blog posts!');
foreach($posts as $post) {
$item = array();
$item = [];
$item['uri'] = $post->find('header a', 0)->href;
$item['title'] = $post->find('header', 0)->plaintext;
$item['content'] = $post->find('div[class="excerpt-content"]', 0)->plaintext;
$item['enclosures'] = array(
$this->getFullSizeImageURI($post->find('div[class*="post-thumb"]', 0)->{'data-original'})
);
// optionally load full articles
if($this->getInput(self::PARAM_BLOG_FULL)) {
$full_html = getSimpleHTMLDOMCached($item['uri']);
$full_html = defaultLinkTo($full_html, $this->host);
$item['author'] = $full_html->find('a[rel="author"]', 0);
$item['content'] = $full_html->find('article', 0);
$item['timestamp'] = strtotime($full_html->find('time.updated', 0)->datetime);
$item['categories'] = $full_html->find('span[class="post_tag"]');
}
$item['uri'] = $post->find('a', 0)->href;
$item['title'] = $post->find('h3', 0)->plaintext;
$item['content'] = $post->find('p', 0)->plaintext;
$item['author'] = $post->find('p', -2)->plaintext;
$item['timestamp'] = strtotime($post->find('p', -1)->plaintext);
// TODO: fetch entire blog post content
$this->items[] = $item;
if($limit > 0 && count($this->items) >= $limit)
if ($limit > 0 && count($this->items) >= $limit) {
return;
}
}
}
@@ -143,53 +120,32 @@ class GlassdoorBridge extends BridgeAbstract {
or returnServerError('Unable to find reviews!');
foreach($reviews as $review) {
$item = array();
$item = [];
$item['uri'] = $review->find('a.reviewLink', 0)->href;
$item['title'] = $review->find('[class="summary"]', 0)->plaintext;
$item['author'] = $review->find('div.author span', 0)->plaintext;
$item['timestamp'] = strtotime($review->find('time', 0)->datetime);
$mainText = $review->find('p.mainText', 0)->plaintext;
// Not all reviews have a title
$item['title'] = $review->find('h2', 0)->plaintext ?? 'Glassdoor review';
$description = '';
foreach($review->find('div.description p') as $p) {
[$date, $author] = explode('-', $review->find('span.authorInfo', 0)->plaintext);
if ($p->hasClass('strong')) {
$p->tag = 'strong';
$p->removeClass('strong');
}
$description .= $p;
$item['author'] = trim($author);
$createdAt = DateTimeImmutable::createFromFormat('F m, Y', trim($date));
if ($createdAt) {
$item['timestamp'] = $createdAt->getTimestamp();
}
$item['content'] = "<p>{$mainText}</p><p>{$description}</p>";
$item['content'] = $review->find('.px-std', 2)->text();
$this->items[] = $item;
if($limit > 0 && count($this->items) >= $limit)
if($limit > 0 && count($this->items) >= $limit) {
return;
}
}
}
private function getFullSizeImageURI($uri) {
/* Images are scaled for display on the website. The scaling takes place
* on the host, who provides images in different sizes.
*
* For example:
* https://www.glassdoor.com/blog/app/uploads/sites/2/GettyImages-982402074-e1538092065712-390x193.jpg
*
* By removing the size information we receive the full sized image.
*
* For example:
* https://www.glassdoor.com/blog/app/uploads/sites/2/GettyImages-982402074-e1538092065712.jpg
*/
$uri = filter_var($uri, FILTER_SANITIZE_URL);
return preg_replace('/(.*)(\-\d+x\d+)(\.jpg)/', '$1$3', $uri);
}
private function filterCompanyURI($uri) {
/* Make sure the URI is a valid review page. Unfortunately there is no
* simple way to determine if the URI is valid, because of automagic

View File

@@ -10,7 +10,9 @@ class GlowficBridge extends BridgeAbstract {
'Thread' => array(
'post_id' => array(
'name' => 'Post ID',
'title' => 'https://www.glowfic.com/posts/<POST ID>',
'title' => 'https://www.glowfic.com/posts/POST ID',
'required' => true,
'exampleValue' => '2756',
'type' => 'number'
),
'start_page' => array(

View File

@@ -10,6 +10,7 @@ class GoComicsBridge extends BridgeAbstract {
'comicname' => array(
'name' => 'comicname',
'type' => 'text',
'exampleValue' => 'heartofthecity',
'required' => true
)
));

View File

@@ -11,19 +11,19 @@ class GogsBridge extends BridgeAbstract {
'global' => array(
'host' => array(
'name' => 'Host',
'exampleValue' => 'https://gogs.io',
'exampleValue' => 'notabug.org',
'required' => true,
'title' => 'Host name without trailing slash',
),
'user' => array(
'name' => 'Username',
'exampleValue' => 'gogs',
'exampleValue' => 'PDModdingCommunity',
'required' => true,
'title' => 'User name as it appears in the URL',
),
'project' => array(
'name' => 'Project name',
'exampleValue' => 'gogs',
'exampleValue' => 'PD-Loader',
'required' => true,
'title' => 'Project name as it appears in the URL',
),
@@ -47,7 +47,7 @@ class GogsBridge extends BridgeAbstract {
'issue' => array(
'name' => 'Issue number',
'type' => 'number',
'exampleValue' => 102,
'exampleValue' => 100,
'required' => true,
'title' => 'Issue number from the issues list',
),

View File

@@ -0,0 +1,68 @@
<?php
class GoogleGroupsBridge extends XPathAbstract {
const NAME = 'Google Groups Bridge';
const DESCRIPTION = 'Returns the latest posts on a Google Group';
const URI = 'https://groups.google.com';
const MAINTAINER = 'Yaman Qalieh';
const PARAMETERS = array( array(
'group' => array(
'name' => 'Group id',
'title' => 'The string that follows /g/ in the URL',
'exampleValue' => 'governance',
'required' => true
),
'account' => array(
'name' => 'Account id',
'title' => 'Some Google groups have an additional id following /a/ in the URL',
'exampleValue' => 'mozilla.org',
'required' => false
)
));
const CACHE_TIMEOUT = 3600;
const TEST_DETECT_PARAMETERS = array(
'https://groups.google.com/a/mozilla.org/g/announce' => array(
'account' => 'mozilla.org', 'group' => 'announce'
),
'https://groups.google.com/g/ansible-project' => array(
'account' => null, 'group' => 'ansible-project'
),
);
const XPATH_EXPRESSION_ITEM = '//div[@class="yhgbKd"]';
const XPATH_EXPRESSION_ITEM_TITLE = './/span[@class="o1DPKc"]';
const XPATH_EXPRESSION_ITEM_CONTENT = './/span[@class="WzoK"]';
const XPATH_EXPRESSION_ITEM_URI = './/a[@class="ZLl54"]/@href';
const XPATH_EXPRESSION_ITEM_AUTHOR = './/span[@class="z0zUgf"][last()]';
const XPATH_EXPRESSION_ITEM_TIMESTAMP = './/div[@class="tRlaM"]';
const XPATH_EXPRESSION_ITEM_ENCLOSURES = '';
const XPATH_EXPRESSION_ITEM_CATEGORIES = '';
const SETTING_FIX_ENCODING = true;
protected function getSourceUrl() {
$source = self::URI;
$account = $this->getInput('account');
if($account) {
$source = $source . '/a/' . $account;
}
return $source . '/g/' . $this->getInput('group');
}
protected function provideWebsiteContent() {
return defaultLinkTo(getContents($this->getSourceUrl()), self::URI);
}
const URL_REGEX = '#^https://groups.google.com(?:/a/(?<account>\S+))?(?:/g/(?<group>\S+))#';
public function detectParameters($url) {
$params = array();
if(preg_match(self::URL_REGEX, $url, $matches)) {
$params['group'] = $matches['group'];
$params['account'] = $matches['account'];
return $params;
}
return null;
}
}

View File

@@ -0,0 +1,61 @@
<?php
class GooglePlayStoreBridge extends BridgeAbstract {
const MAINTAINER = 'Yaman Qalieh';
const NAME = 'Google Play Store';
const URI = 'https://play.google.com/store/apps';
const CACHE_TIMEOUT = 3600; // 1h
const DESCRIPTION = 'Returns the most recent version of an app with its changelog';
const TEST_DETECT_PARAMETERS = array(
'https://play.google.com/store/apps/details?id=com.ichi2.anki' => array(
'id' => 'com.ichi2.anki'
)
);
const PARAMETERS = array(array(
'id' => array(
'name' => 'Application ID',
'exampleValue' => 'com.ichi2.anki',
'required' => true
)
));
const INFORMATION_MAP = array(
'Updated' => 'timestamp',
'Current Version' => 'title',
'Offered By' => 'author'
);
public function collectData() {
$appuri = static::URI . '/details?id=' . $this->getInput('id');
$html = getSimpleHTMLDOM($appuri);
$item = array();
$item['uri'] = $appuri;
$item['content'] = $html->find('div[itemprop=description]', 1)->innertext;
// Find other fields from Additional Information section
foreach($html->find('.hAyfc') as $info) {
$index = self::INFORMATION_MAP[$info->first_child()->plaintext] ?? null;
if (is_null($index)) {
continue;
}
$item[$index] = $info->children(1)->plaintext;
}
$this->items[] = $item;
}
public function detectParameters($url) {
// Example: https://play.google.com/store/apps/details?id=com.ichi2.anki
$params = array();
$regex = '/^(https?:\/\/)?play\.google\.com\/store\/apps\/details\?id=([^\/&?\n]+)/';
if(preg_match($regex, $url, $matches) > 0) {
$params['id'] = urldecode($matches[2]);
return $params;
}
return null;
}
}

View File

@@ -18,7 +18,8 @@ class GoogleSearchBridge extends BridgeAbstract {
const PARAMETERS = array(array(
'q' => array(
'name' => 'keyword',
'required' => true
'required' => true,
'exampleValue' => 'rss-bridge',
)
));

View File

@@ -0,0 +1,107 @@
<?php
class GroupBundNaturschutzBridge extends XPathAbstract
{
const NAME = 'BUND Naturschutz in Bayern e.V. - Kreisgruppen';
const URI = 'https://www.bund-naturschutz.de/ueber-uns/organisation/kreisgruppen-ortsgruppen';
const DESCRIPTION = 'Returns the latest news from specified BUND Naturschutz in Bayern e.V. local group (Germany)';
const MAINTAINER = 'dweipert';
const PARAMETERS = array(
array(
'group' => array(
'name' => 'Group',
'type' => 'list',
'values' => array(
// 'Aichach-Friedberg' => 'bn-aic.de', # non-uniform page
'Altötting' => 'altoetting',
'Amberg-Sulzbach' => 'amberg-sulzbach',
'Ansbach' => 'ansbach',
'Aschaffenburg' => 'aschaffenburg',
'Augsburg' => 'augsburg',
'Bad Kissingen' => 'bad-kissingen',
'Bad Tölz' => 'bad-toelz',
'Bamberg' => 'bamberg',
'Bayreuth' => 'bayreuth', # single entry # different layout
'Berchtesgadener Land' => 'berchtesgadener-land',
'Cham' => 'cham',
// 'Coburg' => 'coburg', # no real entries # different layout
'Dachau' => 'dachau',
'Deggendorf' => 'Deggendorf',
'Dillingen' => 'dillingen',
'Dingolfing-Landau' => 'dingolfing-landau',
'Donau-Ries' => 'donauries',
'Ebersberg' => 'ebersberg',
'Eichstätt' => 'eichstaett', # single entry since 2020
'Erding' => 'erding',
'Erlangen' => 'erlangen',
'Forchheim' => 'forchheim',
'Freising' => 'freising',
'Freyung-Grafenau' => 'freyung-grafenau',
'Fürstenfeldbruck' => 'fuerstenfeldbruck',
'Fürth-Land' => 'fuerth-land',
'Fürth-Stadt' => 'fuerth',
'Garmisch-Partenkirchen' => 'garmisch-partenkirchen',
'Günzburg' => 'guenzburg',
'Hassberge' => 'hassberge',
'Höchstadt-Herzogenaurach' => 'hoechstadt-herzogenaurach',
// 'Hof' => 'kreisgruppehof.bund-naturschutz.com', # non-uniform page
'Ingolstadt' => 'ingolstadt',
'Kelheim' => 'kelheim',
'Kempten' => 'kempten',
'Kitzingen' => 'kitzingen',
'Kronach' => 'kronach',
'Kulmbach' => 'kulmbach',
'Landsberg' => 'landsberg',
'Landshut' => 'landshut',
'Lichtenfeld' => 'lichtenfels',
'Lindau' => 'lindau',
'Main-Spessart' => 'main-spessart',
'Memmingen-Unterallgäu' => 'memmingen-unterallgaeu',
'Miesbach' => 'miesbach',
'Miltenberg' => 'miltenberg',
'Mühldorf am Inn' => 'muehldorf',
// 'München' => 'bn-muenchen.de', # non-uniform page
'Neu-Ulm' => 'neu-ulm',
'Neuburg-Schrobenhausen' => 'neuburg-schrobenhausen',
'Neumarkt' => 'neumarkt',
'Neustadt/Aisch-Bad Windsheim' => 'neustadt-aisch',
'Neustadt/Waldnaab-Weiden' => 'neustadt-weiden',
'Nürnberg Stadt' => 'nuernberg-stadt',
'Nürnberger Land' => 'nuernberger-land',
'Ostallgäu-Kaufbeuren' => 'Ostallgäu-Kaufbeuren',
'Passau' => 'passau',
'Pfaffenhofen/Ilm' => 'pfaffenhofen',
'Regen' => 'regen',
'Regensburg' => 'regensburg',
'Rhön-Grabfeld' => 'rhoen-grabfeld',
'Rosenheim' => 'rosenheim',
'Roth' => 'roth',
'Rottal-Inn' => 'rottal-inn',
'Schwabach' => 'schwabach',
'Schwandorf' => 'schwandorf',
'Schweinfurt' => 'schweinfurt',
'Starnberg' => 'starnberg',
'Straubing-Bogen' => 'straubing',
'Tirschenreuth' => 'tirschenreuth',
'Traunstein' => 'traunstein',
'Weilheim-Schongau' => 'weilheim-schongau',
'Weißenburg-Gunzenhausen' => 'weissenburg-gunzenhausen',
'Wunsiedel' => 'wunsiedel',
'Würzburg' => 'wuerzburg',
),
),
),
);
const XPATH_EXPRESSION_ITEM = '//div[@itemtype="http://schema.org/Article"]';
const XPATH_EXPRESSION_ITEM_TITLE = './/*[@itemprop="headline"]';
const XPATH_EXPRESSION_ITEM_CONTENT = './/*[@itemprop="description"]/text()';
const XPATH_EXPRESSION_ITEM_URI = './/a/@href';
const XPATH_EXPRESSION_ITEM_TIMESTAMP = './/*[@itemprop="datePublished"]/@datetime';
const XPATH_EXPRESSION_ITEM_ENCLOSURES = './/img/@src';
protected function getSourceUrl() {
return 'https://' . $this->getInput('group') . '.bund-naturschutz.de/aktuelles';
}
}

Some files were not shown because too many files have changed in this diff Show More