1
0
mirror of https://github.com/RSS-Bridge/rss-bridge.git synced 2025-08-14 04:24:05 +02:00

Compare commits

...

2613 Commits

Author SHA1 Message Date
Mynacol
5382dee516 [GolemBridge] Fix removal of affiliate images
On
https://www.golem.de/news/anlage-in-etfs-was-alternativen-zum-msci-world-bringen-2508-199041.html
the affiliate box isn't properly filtered out.
The reason seems to be switching from a `div` to an `aside` element.
HTML source fragment:
```html
<aside class="gbox_affiliate" data-nosnippet>
    <div class="gbox_attribution"></div>
    <div class="gbox_fx1">
<a href="https://www.financeads.net/tc.php?t=36731C67231788T" target="_blank" rel="nofollow" onclick="_gcpx.push(['ev','d','rklmbox/14387']); return true;"><img src="https://scr3.golem.de/screenshots/affiliate/14
387/9caaa476f979dcf7457395f39ac9ed9f.png" alt=""></a>
        <div class="gbox_fx2">
            <div class="gbox_title">Tagesgeld, Festgeld, ETFs, Aktien und mehr bei raisin</div>
            <div><a class="gbox_btn" data-cta="Jetzt Investmentm&ouml;glichkeiten bei raisin entdecken" href="https://www.financeads.net/tc.php?t=36731C67231788T" target="_blank" rel="nofollow" onclick="_gcpx.push(['ev','d','rklmbox/14387']); return true;"></a></div>
        </div>
    </div>
<!-- /gbox --></aside>
```
2025-08-13 12:15:26 +02:00
Mynacol
b60556ffb4 [HeiseBridge] Remove "Videos by heise" ads
This seems to be a new middle-of-content self-ad.
Seen on https://heise.de/-10519045

The code snippet in that case was:
```html
<div class="ad ad--inread">

  <div class="ad--inread-header">
    <p class="ad--inread-header__text">
      Videos by heise
    </p>

    <div class="ad--inread-header__more">
      <button class="ad--inread-header-menu-toggle" popovertarget="ad--inread-header-menu">
        mehr Videos
        <svg fill="none" height="24" viewbox="0 0 24 24" width="24" xmlns="http://www.w3.org/2000/svg">
          <path d="M8.625 12.0023C8.625 12.2094 8.45711 12.3773 8.25 12.3773C8.04289 12.3773 7.875 12.2094 7.875 12.0023C7.875 11.7952 8.04289 11.6273 8.25 11.6273C8.45711 11.6273 8.625 11.7952 8.625 12.0023ZM8.625 12.0023H8.25M12.375 12.0023C12.375 12.2094 12.2071 12.3773 12 12.3773C11.7929 12.3773 11.625 12.2094 11.625 12.0023C11.625 11.7952 11.7929 11.6273 12 11.6273C12.2071 11.6273 12.375 11.7952 12.375 12.0023ZM12.375 12.0023H12M16.125 12.0023C16.125 12.2094 15.9571 12.3773 15.75 12.3773C15.5429 12.3773 15.375 12.2094 15.375 12.0023C15.375 11.7952 15.5429 11.6273 15.75 11.6273C15.9571 11.6273 16.125 11.7952 16.125 12.0023ZM16.125 12.0023H15.75M21 12.0023C21 16.9729 16.9706 21.0023 12 21.0023C7.02944 21.0023 3 16.9729 3 12.0023C3 7.03176 7.02944 3.00232 12 3.00232C16.9706 3.00232 21 7.03176 21 12.0023Z" stroke="#777" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5"></path>
        </svg>
      </button>

      <div class="ad--inread-header-menu" id="ad--inread-header-menu" popover>
        <ul class="a-u-mb-0">
          <li>
            <a class="ad--inread-header-menu-link" href="https://www.youtube.com/@ct3003" target="_blank">
              c&#39;t 3003
            </a>
          </li>
          <li>
            <a class="ad--inread-header-menu-link" href="https://www.youtube.com/heiseonline" target="_blank">
              heise &amp; ct
            </a>
          </li>
          <li>
            <a class="ad--inread-header-menu-link" href="https://peertube.heise.de/" target="_blank">
              Peertube
            </a>
          </li>
        </ul>
      </div>
    </div>
  </div>

  <figure class="video video--fullwidth">
    <a-video entry-id="25969" height="9" instant is-target-video-playlist style="aspect-ratio: 16 / 9" type="targetvideo" width="16"></a-video>
  </figure>
</div>
```

Hence filtering anything with the class `ad` or `ad--inread` gets rid of
it.
2025-08-12 21:25:42 +02:00
Dag
37174f01e5 fix: throw client exception in some bridges (#4661) 2025-08-08 02:24:13 +02:00
Dag
a599f4ba83 fix: dont log user errors (#4660) 2025-08-08 02:16:43 +02:00
Dag
81ce9c9483 fix: introduce system env var, remove debug mode (#4658)
* fix: introduce system env var

* docs

* docs
2025-08-08 01:38:12 +02:00
Dag
a128c05a97 docs: emphasize strict types (#4657) 2025-08-05 21:06:40 +02:00
Dag
9caa043fe1 lint: add returnClientError and returnServerError to forbiddenFcuntions (#4656) 2025-08-05 20:55:04 +02:00
Dag
f11571ae78 refactor: rename functions (#4655)
returnClientError => throwClientException
returnServerError => throwServerException

New convenience function: throwRateLimitException

Old functions are kept but deprecated.
2025-08-05 20:44:40 +02:00
Dag
b39964cee3 chore: prepare for aug 2025 release (#4654) 2025-08-05 19:50:27 +02:00
Joseph
9c43921a33 [FirstLookMediaTechBridge] Remove bridge (#4653)
Website no longer exists
2025-08-04 22:57:35 +02:00
Joseph
9e2975048f [AskfmBridge] Remove bridge (#4652)
Website closed in December 2024 https://web.archive.org/web/20241129120541/https://about.ask.fm/closure-notice-the-platform-to-be-deactivated-december-1-2024/
2025-08-04 22:56:27 +02:00
Joseph
fb153f9a92 [DansTonChatBridge] Remove bridge (#4650)
bridge is broken and website has native feeds.

https://danstonchat.com/category/quote/feed
2025-08-04 17:19:24 +02:00
Joseph
20fec74c63 [DailymotionBridge] Fetch playlist title from API (#4649) 2025-08-04 15:41:04 +02:00
Simone Dotto
b5f90f8d47 [AmazonPriceTracker] Fix price not shown, new default source (#4631)
Fixes issue #4586

Co-authored-by: Simone Dotto <simonedotto@proton.me>
2025-08-04 14:31:43 +02:00
shaun
aba38845d2 [YoutubeCommunityTabsBridge] Rename Community→Posts to fix broken bridge (#4606)
* youtube community posts are just called "Posts" now

* finish renaming Community -> Posts

* add feedName fallbacks (thanks @Mar-Koeh)

* rename YouTubePostsTabBridge back to YouTubeCommunityTabBridge

* fix linter error by breaking up long expression

* fix optional-chaining regression by using ‘?? null’
2025-08-04 14:30:48 +02:00
Joseph
1211ac63d9 Update DailymotionBridge.php (#4648) 2025-08-04 14:28:16 +02:00
Joseph
640503168e [FirefoxAddonsBridge] Minor change to item content html (#4647) 2025-08-04 14:27:40 +02:00
Arnav Jain
93de253d01 [GoComicsBridge] cache individual comic page for 24h (#4646) 2025-08-04 14:27:19 +02:00
User123698745
6ec4da854f [FallGuysBridge] fix: handle new data structure (#4640)
* [FallGuysBridge] fix: handle new data structure

* [FallGuysBridge] review feedback: removed mixed
2025-08-04 01:36:44 +02:00
Dag
e5f9fe6251 lint (#4645) 2025-08-04 01:36:15 +02:00
Dag
47c9983e16 fix: dont cache basic auth response (#4644) 2025-08-04 01:32:36 +02:00
Sandro
69eda522c8 Mention php extension filter (#4608)
While trying around to minimize my installation, I noticed that this
extension is nowhere mentioned.
2025-08-04 01:09:38 +02:00
User123698745
172e7eb280 [prtester] fix wrong pr check fail when refactoring code (the bridge html output has not changed) (#4642)
ignore "nothing to commit, working tree clean"
2025-08-04 01:08:25 +02:00
User123698745
acb9373c10 [DRKBlutspendeBridge] add offers to content & add caption to images & use cached request (#4641) 2025-08-04 01:07:41 +02:00
Joseph
85497238c5 Update HaveIBeenPwnedBridge.php (#4638) 2025-08-04 00:58:09 +02:00
Marcin Morawski
a2334838a6 Fix deprecations (#4636)
* Fix PHP 8.4 deprecation

Implicitly marking parameter as nullable is deprecated, the explicit nullable type must be used instead

* [github workflow] Add additional php versions
2025-08-04 00:55:50 +02:00
mruac
c65fbd5543 [BlueskyBridge] Fix cases for missing reply post context and QoL fix for video loading (#4635)
* added fix for missing reply post context

* qol fix - no preload on videos
2025-08-04 00:50:12 +02:00
sysadminstory
e241f3dcde [PepperBridgeAbstract, DealabsBridge, HotUKDealsBridge, MydealsBridge] Adapt RSS bridge to website content update; remove country of origin due to missing data (#4634)
Website use now "vue3" and some class and attributes have changed their
names : bridge was updated to use the new class and attribute names

Country of origin has been removed from the deal list : it's for now
disabled, but code is still present in the bridge, in case the website
enable it again.
2025-08-04 00:48:27 +02:00
Pavel Korytov
16bb6156a5 [UniverseTodayBridge] Add bridge (#4627) 2025-08-04 00:22:50 +02:00
Pavel Korytov
9f8dc411a4 [InstituteForTheStudyOfWarBridge] Increase caching time (#4626) 2025-08-04 00:21:57 +02:00
July
5b97899734 [FanaticalBridge] Create a new bridge (#4624)
Provides a fairly barebones bridge for Fanatical bundles:
- Tags detail bundle tiers and prices
- Contents name and link to each bundle item
- Images for each item are in enclosures
2025-08-04 00:21:04 +02:00
July
8ae2c2e3c3 [HumbleBundleBridge] Overhaul to include more information (#4621)
* [HumbleBundleBridge] Overhaul to include more information

* [HumbleBundleBridge] Remove use of named args in calls

PHP 7.4 lacks named arg support and fails unit tests
2025-08-04 00:20:00 +02:00
July
9ec6ae39a2 [ComickBridge] Add new bridge (#4625)
Makes new brige for manga from comick.io. Like the CubariProxyBridge,
can provide manga page images in feed entry content or enclosures.
2025-08-04 00:19:08 +02:00
July
3517cda4a5 [YouTubeFeedExpanderBridge] More reliable channel icons (#4622) 2025-08-04 00:17:30 +02:00
July
52be29d3ec [AnnasArchiveBridge] Fix book list CSS selector (#4619) 2025-08-04 00:17:01 +02:00
July
696aed22cc [CubariProxyBridge] Replace MangaSee with WeebCentral (#4618) 2025-08-04 00:16:30 +02:00
July
e394be7ca5 [KemonoBridge] Add search query support (#4620) 2025-08-04 00:16:14 +02:00
jaydeethree
3835f290c1 Update GOGBridge to use GOG's REST API. I have tested this locally and it seems to work correctly. (#4616) 2025-08-04 00:14:51 +02:00
Nomis
c7de5c95be Update 06_Public_Hosts.md (#4614)
Remove bridge.easter.fr
2025-08-04 00:12:38 +02:00
Tobias Alexander Franke
71808aaa81 [WarhammerComBridge] Bridge for Warhammer Community blog (#4610)
* [WarhammerComBridge] Bridge for Warhammer Community blog

* Fix Linter issues
2025-08-04 00:10:58 +02:00
Anton Smirnov
2ca696c1cf [EpicGamesFreeBridge] productSlug can be null; also add a universal future-proof-ish fallback (#4595)
* productSlug can be null, do more discovery, add fallback

* productSlug can be garbage too, remove it completely
2025-08-03 23:59:42 +02:00
Sebastian K
c90b98b965 Error handling in ExplosmBridge (#4600)
Skip further processing if element was not found to avoid errors
2025-08-03 23:58:24 +02:00
Quentin B.
8e880de3d2 [CentreFranceBridge] Fix parser following website update (#4596)
* [CentreFranceBridge] Fix parser following website update

* [CentreFranceBridge] Fix empty content

* [CentreFranceBridge] Fix title parsing
2025-08-03 23:52:06 +02:00
Tone
bfa6c4c080 [HeiseBridge] removes language-info-text, add archive.is link for people without subscription (#4594)
* [HeiseBridge] removes language-info-text, add archive.is link for people without subscription

* fix annoying phpcs
2025-08-03 23:50:54 +02:00
User.
5ab938ada7 [WaggaCouncilBridge] Add bridge (#4593)
Co-authored-by: Scrub000 <scrub@example.com>
2025-08-03 23:49:10 +02:00
Petr Prenghy
4d2fe2f12d [NasestrechaBridge] Add bridge (#4591)
* Add files via upload

Bridge for NaseStrecha.cz - NaseStrecha.cz is a specialized Czech news and advice portal focusing on roofs, construction, and home improvement, offering reliable expert guidance on roofing materials, insulation, and energy-saving techniques nasestrecha.cz . It is run by the team behind the Strechy-Solar-Remeslo trade fair and includes up-to-date news, practical tips, and industry events

* phpcs fix

* Bridge for i4wifi.cz for product news.
The website i4wifi.cz is a wholesale distributor specializing in wireless, networking, and photovoltaic equipment, offering products from brands like MikroTik, Ubiquiti, and Hikvision. It provides a wide range of network solutions, technical support, and training services for businesses and professional installers in the Czech Republic and beyond.
2025-08-03 23:46:35 +02:00
Mynacol
4c0b97d605 [ZeitBridge] Add advertorial marker to article
So users are aware that it's a paid article.

Some might still find them interesting, so we cannot just filter them
away.
2025-07-20 01:35:28 +02:00
Mynacol
1d5bcba41f [ZeitBridge] Hide magazine ads in articles
Test article: https://www.zeit.de/campus/2025/03/kyoto-university-abschlussfeier-kostueme-japan
2025-07-20 01:35:28 +02:00
Mynacol
d19ce75d4b Merge pull request #4613 from Mynacol/golem-add-table
[GolemBridge] Add tables to content
2025-07-16 13:53:53 +02:00
Mynacol
bfbe2abdce [GolemBridge] Add tables to content
For example the following article has such tables that should be
included:
https://www.golem.de/news/immobilien-mieten-oder-kaufen-warum-es-dabei-nicht-nur-ums-geld-geht-2507-197406.html
2025-07-16 11:50:00 +00:00
Jonathan Kay
354cea09a7 [GoComicsBridge] Add fallback when link to current comic is missing (#4589) 2025-06-08 21:57:41 +02:00
sysadminstory
8dada08e69 [IdealoBridge] Bypass bot protection (#4588)
Add some headers (User-Agent, Accept, Accept-Language) and activate
compression to bypass the bot protection
2025-06-07 23:31:02 +02:00
Jonathan Kay
514b3edf0b [GoComicsBridge] Fix for JSON being removed (#4585)
- Now redirects to first comic from landing page
- Switched to meta tags
2025-06-05 23:41:20 +02:00
Tobias Alexander Franke
7aa54602cf [FabBridge] Pull 100% discounted items via Fab API (#4584)
* [FabBridge] Pull 100% discounted items via Fab API

* [FabBridge] Linter fixes
2025-06-04 22:15:28 +02:00
Dag
98e03011db chore: prepare for 2025-06-03 release (#4583) 2025-06-03 21:24:35 +02:00
Anton Smirnov
b8064d9dfe [EpicGamesFree] Fixes: url not set, other promos shown (#4575)
* URI was not set because of the typo

* Filter out other promos
2025-05-30 11:05:36 +02:00
Mynacol
976217111c [GolemBridge] Add code elements
The extractor missed <pre> elements for code snippets.
For example the code line in
https://www.golem.de/news/falsch-deklarierte-hdds-betrug-bei-festplatten-bleibt-ein-problem-2505-196675.html
2025-05-28 21:21:44 +02:00
Joseph
419844f010 Delete OpenlyBridge.php (#4572) 2025-05-26 22:46:42 +02:00
Joseph
e5b3ec85d9 Delete CuriousCatBridge.php (#4571) 2025-05-26 22:46:28 +02:00
Stéphane
7b55eb3824 Adding a bridge for Paul Graham's essays (#4570)
* Adding a bridge for Paul Graham's essays

* lint

---------

Co-authored-by: Dag <me@dvikan.no>
2025-05-25 20:46:50 +02:00
Dag
7397cabeee fix(telegram): remove meta message (#4569) 2025-05-24 19:29:04 +02:00
Thiago Ferreira
daef06c6dd devcontainer: Fixed Dev Containers setup (#4556)
The current setup for Dev Containers was not working, with multiple
different errors. So, in order to restore its funcionality (and allow
for things like linting and debugging), the following changes were made:

- The Dockerfile was severely alterered. Now, the `docker-php-ext-enable` binary is installed before its usage,
  it points to the correct PHP binary, and we install Composer for for
  loading dev-dependencies later-on.

- Moved the "postCreateCommand" section (defined on the `devcontainer.json` file) into its own script file (for a
  more readable experience)

- On the post-creation script, moved the `xdebug.ini` to the correct
  directory (alongside the PHP-FPM bin), installed PHPUnit,
  PHPCodesniffer (and the 'PHP Compatibility' sniffer) with Composer on
  a global location, and changed owner of the `cache` directory

- Changed VSCode-specific customization setting in order to point to the
  update some binary paths. Also made sure globally-installed composer
  packages binaries are accessible via PATHdocker-php-ext-enable
2025-05-24 19:18:52 +02:00
Dag
ec5b32c551 ci: fix broken ci (#4568)
* fix: deprecation warning

* ci: fix broken ci
2025-05-24 19:14:53 +02:00
Dag
0130adcd6c fix: deprecation warning (#4567) 2025-05-23 22:55:41 +02:00
Dawid Wróbel
b7c04f8587 Overhaul the usage of libcurl-impersonate (#4535)
libcurl-impersonate was not being used properly, as the code was
overriding the headers set by it to prevent detection.

- update the libcurl-impersonate to an actively managed lexiforest
  fork
- impersonate Chrome 131
- move the defaultHttpHeaders to http.php, where it belongs
- only set defaultHttpHeaders if curl-impersonate is not detected
- make useragent ini setting optional and disabled by default
- add necessary documentation updates
2025-05-17 20:18:36 +02:00
Christian Schabesberger
0f77d3ae0a fix nnplus article filter (#4555) 2025-05-10 21:48:54 +02:00
Dag
8f21a030a8 fix(furaffinity): type error (#4554)
fixes array_filter(): Argument #1 ($array) must be of type array, null given

fix #4553
2025-05-09 09:39:35 +02:00
Dag
d36b335725 fix: do not log rate limit exceptions (#4552) 2025-05-09 06:14:13 +02:00
Dag
b8c0c1f3b8 fix: tweak logging rules (#4551) 2025-05-09 05:58:11 +02:00
tillcash
fd267df0e9 [LinuxBlogBridge] fix typo (#4549) 2025-05-09 05:41:10 +02:00
Dag
6c4225441a fix(tiktok) (#4550) 2025-05-09 05:40:48 +02:00
Apollo Nargang
5bd767b862 [TikTokBridge] Use oEmbed for video metadata (#4514)
* [TikTokBridge] Use oEmbed for video metadata

Fetches oEmbed-formatted metadata for videos through the TikTok API to
provide post titles, thumbnails, and authors. This hasn't yet been
tested, so it's possible it doesn't work.

* [TikTokBridge] Add back view count parsing

oops

* [TikTokBridge] Prepend www to the oEmbed API endpoint URL

The non-www URL resulted in a 301 redirect to the www URL, so this just
skips that redirect, improving performance a bit and hopefully helping
with the 400 errors.

* [TikTokBridge] Retry failed OEmbed requests

If an OEmbed request fails, retry a few times, waiting a bit in between
each retry. This should fix the problem for the most part, since I think
the problem was related to some sort of rate limit (it isn't mentioned
in the docs, but it seems to only happen when sending large quantities
of sequential requests).
2025-05-09 05:10:04 +02:00
Dawid Wróbel
72e1998e16 [AllegroBridge]: fix, use JSON instead of HTML (#4536)
Cookie is now obligatory, otherwise 403 is returned
2025-05-09 05:06:23 +02:00
Tone
083ba1e4f7 [FinanzflussBridge] fix for images not displayed (#4538) 2025-05-09 04:36:22 +02:00
Jonathan Kay
1cb9e91697 [GoComicsBridge] Update fix for latest layout changes (#4539) 2025-05-09 04:35:59 +02:00
sysadminstory
6342b8387e [InstagramBridge] Use fallback when User ID can not be found (#4531)
- In case the userId can not be found, use the Fallback method

- Fallback method move to it's own function
2025-05-09 04:33:18 +02:00
tillcash
648fcc38b5 [LinuxBlogBridge] add bridge (#4528)
* [LinuxBlogBridge] add bridge

* refactor

* Update LinuxBlogBridge.php
2025-05-09 04:28:31 +02:00
√(noham)²
9fb4a5dd72 Apple App Store bridge fix (#4516)
* Apple App Store bridge fix

* Fixe AppleAppStore + lint

* fix endpoint
2025-05-09 03:33:56 +02:00
Dag
83edf5a48b fix(CssSelector): html entity decode bug, fix #4484 (#4547) 2025-05-09 03:26:10 +02:00
Dag
66f1d449a7 test (#4546) 2025-05-09 02:15:28 +02:00
Petr Prenghy
908937383b [ElektroARGOSBridge] add new bridge - News, events and promotions on ARGOS electro shop (#4523) 2025-05-09 00:23:21 +02:00
Dag
67c5198cbb chore(fdroid): remove dead bridge (#4545) 2025-05-09 00:15:48 +02:00
Dag
9dc673a038 fix(github): PRs and issues (#4544) 2025-05-09 00:09:28 +02:00
Dag
58e30f8b4b fix(furaffinity): date and tags, #4513 (#4543) 2025-05-08 23:33:18 +02:00
Dag
e6a84052f0 fix(reddit): handle absent search keywords, #4502 (#4542) 2025-05-08 23:04:12 +02:00
Dag
e364dd1a20 fix(atom): omit item timestamp if absent (#4541)
prev behavior inserted current time, which seems wrong
2025-05-08 22:37:56 +02:00
tillcash
e69ceba237 [ZonebourseBridge] Add Bridge (#4501) 2025-05-08 22:15:55 +02:00
Dag
0d20a8c48c fix(telegram): trim username for convenience #4520 (#4521) 2025-04-16 02:47:57 +02:00
Petr Prenghy
a6ee840533 Update 06_Public_Hosts.md (#4519)
new mirror in The Czech Republic
2025-04-14 12:55:41 +02:00
Dag
95af1ffddf fix(reuters): tweak, try to avoid antibot (#4515) 2025-04-08 21:12:42 +02:00
July
d6a9da1cc8 [SubstackProfileBridge] Add new bridge (#4507) 2025-04-03 07:51:58 +02:00
Jonathan Kay
85962e18d3 [GoComicsBridge] New layout fix and added features (#4510)
* Updated to use the new layout launched April 1st
* Adds new title date/full name option
* Adds limit option for how many days of comics to get
2025-04-03 07:50:16 +02:00
July
a19b63e840 [AO3Bridge] Add option to make one entry per fic (#4508) 2025-04-02 04:09:28 +02:00
tillcash
5365b57638 [MinecraftBridge] fix favicon (#4506) 2025-04-02 03:57:40 +02:00
Dag
462c005f2c fix: dont read /etc if open_basedir #4502 (#4505) 2025-04-01 01:15:59 +02:00
ORelio
db42f2786c [FeedExpander] Add prepareXml() overridable function (#4485)
* FeedExpander: Remove tailing content in XML

- Move preprocessing code into overridable preprocessXml()
- Auto-remove trailing data after root xml node

* FeedExpander: Add PR reference with use case

* FeedExpander: Code linting

* [FeedExpander] Keep content at end of document for now

Will add back later if more sites have the same issue

* [FeedExpander] prepareXml: Add type hints
2025-04-01 00:42:08 +02:00
ORelio
26a4c255d3 [html] convertLazyLoading: Add parseSrcset() (#4503)
* [html] convertLazyLoading: Add parseSrcset()

Add srcset parser closer to the specifications

* [html] code linting

* [html] parseSrcset: Add type hints, check preg_match_all
2025-04-01 00:41:33 +02:00
subtle4553
3055e69c23 [ManyVidsBridge] Fix parsing of URL input (#4499) 2025-03-27 21:02:12 +01:00
tillcash
7c1e01b45a [MinecraftBridge] Add Bridge (#4497) 2025-03-26 19:46:02 +01:00
Dag
4d8a46d46e feat: add sanity check for required curl module (#4495) 2025-03-26 00:07:33 +01:00
Dag
9d6aa5ee38 fix: operator precedence bug (#4494) 2025-03-25 23:52:47 +01:00
subtle4553
1c45eff505 [ManyVidsBridge] Create proper feed content (#4493) 2025-03-25 23:34:19 +01:00
Joseph
68ff39e164 [TheFarSideBridge] Remove hotlink protection bypass (#4492) 2025-03-25 21:55:09 +01:00
mruac
abb1602524 fix #4475 (#4491)
* support embeds for feeds, lists and starter packs

* lint
2025-03-25 21:54:25 +01:00
Pavel Korytov
87112497de [AnthropicBridge] Delete bridges (#4490) 2025-03-25 21:52:53 +01:00
Niehztog
38bb5115c9 fix issues reported in https://github.com/RSS-Bridge/rss-bridge/issues/4477 (#4488) 2025-03-24 21:12:26 +01:00
Tomasz Molski
23cb9349fc [CeskaTelevizeBridge] Adjusted getting article timestamp (#4486)
* [CeskaTelevizeBridge] Adjusted getting article timestamp

* [CeskaTelevizeBridge] Removed excess whitespace
2025-03-23 21:30:45 +01:00
Pavel Korytov
05a9ac0f06 [OpenCVEBridge] Rewrite for API change (#4476)
* [OpenCVEBridge] Rewrite for API change

* [OpenCVEBridge] Fix lint
2025-03-23 21:01:21 +01:00
Dan Wainwright
91fe6c1fae [BazarakiBridge] Add new bridge (#4473)
* [BazarakiBridge] Add new bridge

* fix

---------

Co-authored-by: Dag <me@dvikan.no>
2025-03-23 20:57:17 +01:00
chibicitiberiu
7260f28e10 [RedditBridge] Added time interval and filter for min comment count (#4471)
* Reddit Bridge - added filter for min comment count and time interval.

* [RedditBridge] Add sort by comment count

* lint

* consistent commas

---------

Co-authored-by: Dag <me@dvikan.no>
2025-03-23 20:45:35 +01:00
Tomasz Molski
87ab1e4513 [BruegelBridge] Initial commit (#4470) 2025-03-23 19:50:11 +01:00
André Andersson
dee734d360 Add Auctionet bridge (#4452) 2025-03-05 19:41:24 +01:00
Latz
744f996224 Added bridge for Toms Touché (https://taz.de/#!tom=tomdestages) (#4438) 2025-03-05 19:39:18 +01:00
Pavel Korytov
f270cd35e7 [TldrTechBridge] Fix duplicate entries and empty sections (#4466) 2025-03-05 19:36:41 +01:00
Tomasz Molski
83c36a87e2 [ReutersBridge] Adjust Fact Check feed path (#4465) 2025-03-05 19:35:12 +01:00
Tomasz Molski
810e17b556 feat: added LeagueOfLegendsNewsBridge (#4462) 2025-03-05 19:34:35 +01:00
sysadminstory
97f07cf216 [InstagramBridge] Add a fallback to the "Username" mode (#4461)
- Added some header that could help Instagram to not block RSS Bridge
- Added a fallback function to use the "Embed profile" Instagram feature
  to get the content shared by one Instagram user
2025-03-05 19:32:03 +01:00
sysadminstory
62fafdc24b [FreeTelechargerBridge] Update URL and some fix (#4459)
- Updated the URL to the new URL in the bridge Meta Data
- Use an other URL that seems to permit to bypass CF protection
  (sometimes)
2025-03-05 19:30:38 +01:00
sysadminstory
cd4cdcfd65 [RadioMelodieBridge] Fix media content (#4458)
- Fix the audio source with the absolute URL
- Fix the pictture enclosure URL (those are already absolute URL)
2025-03-05 19:30:09 +01:00
Tobias Alexander Franke
00a24e2f69 New bridge for the latest Shadertoy submissions (#4456)
* New bridge for the latest Shadertoy submissions

* [ShadertoyBridge] Linter fixes

* [ShadertoyBridge] More Linter fixes

* [ShadertoyBridge] Even more Linter fixes
2025-02-26 10:20:28 +01:00
André Andersson
92b5e7093f Fix data-lot-id not being correctly set so use href instead (#4453) 2025-02-24 17:58:24 +01:00
Dag
b52f01505d fix(github): semi-repair (#4449) 2025-02-14 02:42:23 +01:00
Dag
e4c32bb046 fix(vk): semi-disable broken bridge (#4448) 2025-02-14 02:00:07 +01:00
Christian Schabesberger
dd4dcfa59c fix nn.de description and paywall filter (#4444) 2025-02-08 01:41:51 +01:00
Tostiman
4e678c955f fix CarThrottleBridge (#4442) 2025-02-05 18:41:42 +01:00
July
549bed64d2 [YouTubeFeedExpanderBridge] Add bridge (#4430) 2025-02-04 20:11:43 +01:00
sysadminstory
94924d8e16 [PepperBridgeAbstract, DealabsBridge, HotUKDealsBridge, MydealsBridge] Fix parameters typo (#4439)
Fixed typo in DealabsBridge and HotUKDealsBridge parameters name
2025-02-03 23:24:42 +01:00
sysadminstory
920b21b1fd [PepperBridgeAbstract, DealabsBridge, HotUKDealsBridge, MydealsBridge] Fixing bridge and add subcategories (#4436)
- Follow site change to get deal data (fix for #4432)
- Add Categories (sub categories in reality) support
2025-02-03 15:35:48 +01:00
Dag
935075072b fix: set default cache ttl of 1d (#4434) 2025-01-30 21:05:17 +01:00
July
3ae7a10223 [GovTrackBridge] Rebase on top of official RSS feed (#4429) 2025-01-29 11:11:25 +01:00
Tone
bf431a6eae [AnisearchBridge] changed id of div so trailers work again (#4428) 2025-01-27 21:55:34 +01:00
Dag
824ac5e373 docs (#4427)
* docs

* docs
2025-01-26 21:24:33 +01:00
Bartosz Sosna
ae8394d976 Fix lfc.pl bug with page content when comments exist (#4425)
* Add lfc.pl bridge

* Adjust bridge

* Add comments section

* Fix a bug with page content when comments exist

* Add brtsos to CONTRIBUTORS.md
2025-01-26 18:58:03 +01:00
Dag
4da61b7922 chore: prepare 2025-01-26 release (#4424) 2025-01-26 11:16:35 +01:00
burrow335
8b1ba003a8 Add support for custom feeds in posts (#4413) 2025-01-25 18:46:12 +01:00
Bartosz Sosna
230edf602e Add lfc.pl bridge (#4419)
* Add lfc.pl bridge

* Adjust bridge

* Add comments section
2025-01-25 18:43:27 +01:00
Eugene Molotov
bd7d1734c3 [RutubeBridge] Use publication time instead of creation time (#4417)
Publication time is shown in video page itself, so it is more essential
2025-01-25 18:40:13 +01:00
Dag
dd8bc077ed feat(FeedParser): recursively parse rss modules (#4422)
Also stop excluding the media module

fix #4415
2025-01-25 18:29:01 +01:00
SebLaus
952a2d99a3 Beginning of URL not needed anymore: ErrorMessage: cURL error Could not resolve host: www.bundestag.dehttps: 6 (https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://www.bundestag.dehttps://www.bundestag.de/parlament/praesidium/parteienfinanzierung/fundstellen50000/2025/2025-inhalt-1032412 (#4420) 2025-01-25 18:28:36 +01:00
Dag
58b3cfb158 fix: drop extension requirement in feed icon url, fix #4416 (#4421) 2025-01-25 17:43:03 +01:00
Eugene Molotov
028acd0af1 [VkBridge] Unassign maintainer (#4418) 2025-01-25 17:27:36 +01:00
axor-mst
2a58f82bd8 [Formula1Bridge] API key and URL format update (#4412)
* [Formula1Bridge] API key and URL format update

* [WorldCosplayBridge] Bridge removal
2025-01-20 17:32:41 +01:00
Simon Alberny
5214581386 Fix MondeDiplo empty date (#4407) 2025-01-15 20:50:56 +01:00
Sebastian Wolf
eadea242a7 [FragDenStaatBridge] remove bridge, site provides full feed at fragdenstaat.de/artikel/feed/ (#4405) 2025-01-12 17:03:27 +01:00
Pavel Korytov
1a2c1f5bba [OllamaBridge] Add bridge (#4403)
* [OllamaBridge] Add bridge

* [OllamaBridge] Fix typo
2025-01-10 20:28:58 +01:00
vdbhb59
776a1f47f3 Update 06_Public_Hosts.md (#4401)
Updated my hosting provider & country to reflect the correct details.
2025-01-10 13:08:35 +01:00
Tone
39ecd63f72 [GolemBridge.php] changed cookie (#4399)
the cookie value changed, without the new cookie it's not possible to parse the articles
2025-01-07 23:40:55 +01:00
Pavel Korytov
0e2655fc8a [AnthropicBridge] Add Anthropic Bridge (#4398)
* [AnthropicBridge] Add Anthropic Bridge

* [AnthropicBridge] Fix lint
2025-01-06 19:10:12 +01:00
Pavel Korytov
e355276378 [EconomistWorldInBriefBridge] Update bridge (#4397)
* [EconomistWorldInBriefBridge] Fix and update bridge

* [EconomistWorldInBriefBridge] Fix lint
2025-01-06 19:08:08 +01:00
Dag
cb65125dbd feat: add section link to frontpage bridge card (#4396) 2025-01-04 20:34:36 +01:00
Dag
1d02214e12 feat: extract simple_html_dom max_file_size to config (#4395) 2025-01-04 19:43:48 +01:00
Dag
48cb7d71ed feat(telegram): add pagination fetching of messages (#4394)
* feat(telegram): add pagination fetching of messages

* docs
2025-01-04 19:00:26 +01:00
Dag
f9e9c8101e Fix 257 (#4393)
* fix(tldrtech): trim duplicate leading slashes

* fix
2025-01-03 08:41:55 +01:00
Dag
97f7df0d06 feat(feedmerge): remove duplicates based off of title too (#4392) 2025-01-03 08:17:47 +01:00
Dag
db3899f2e6 fix(legifrance): emergency repair, still semi-broken (#4391) 2025-01-03 07:23:13 +01:00
Dag
d36cd0a332 fix(ceska): item image (#4390) 2025-01-03 07:11:08 +01:00
Dag
662e0bfa95 refactor(donnons) (#4389) 2025-01-03 06:49:10 +01:00
Dag
3fc38c15a3 fix: cache 400 and 404, and refactor token auth (#4388)
* fix(cache): also cache 400 and 404 responses

* refactor(token_auth)
2025-01-03 06:19:24 +01:00
Dag
be51ba17df fix(url): disallowed wonky path (#4386) 2025-01-03 05:40:30 +01:00
Dag
c44a76ff17 refactor: remove dead code (#4385) 2025-01-03 05:04:49 +01:00
Dag
7c6d4a932c fix: upgrade hardcoded version number, fix #4382 (#4384) 2025-01-03 01:58:38 +01:00
Sebastian Wolf
45ee018a6e [MixologyBridge] add null checks for author and timestamp elements (#4383)
* [MixologyBridge] add null checks for author and timestamp elements

* [MixologyBridge] fix formatting
2025-01-03 01:43:39 +01:00
Dag
e825272987 fix(rumble): exterminate double leading slashes in item url (#4381)
Fixed for items with pub date newer than 31. jan 2025
2025-01-02 18:22:47 +01:00
Niehztog
97eebfb562 [BlizzardNewsBridge] fix BlizzardNewsBridge (#4379)
* fix BlizzardNewsBridge

* fix linter warnings

* fix linter warnings

* fix linter warnings
2025-01-02 17:44:36 +01:00
mruac
2a44a006b2 Update BlueskyBridge.php (#4367)
* Update BlueskyBridge.php

* Used human readable terms
* Include quote and reply post
* Added video support
* Replaced Youtube embed with thumbnail preview
* Added link embed preview
* Included visible alt text to images

* appease the lint

* remove unused test code

* fix unset displayName

* appease the lint
2025-01-02 17:39:07 +01:00
Sebastian Wolf
974f00cd6a [MixologyBridge] adapt to latest site changes (#4368)
* [MixologyBridge] adapt to latest site changes

* [MixologyBridge] fix category selector
2025-01-02 17:17:54 +01:00
Quentin B.
4b4d622333 [CentreFranceBridge] Update parser to handle latest website layout changes (#4372) 2025-01-02 17:14:10 +01:00
Florent V.
b4a63e7040 [EdfPrices Bridge] add HC/HP, base and EJP (#4369)
* [EdfPrices Bridge] add HC/HP, base and EJP

* [EdfPrices Bridge] lint

* [EdfPrices Bridge] fix missing variable
2025-01-02 16:45:33 +01:00
Dag
7d544f1fab feat(reddit): support video (#4380) 2025-01-02 16:33:56 +01:00
Dag
152e96d3d0 fix: broken if_not_modified_since (#4377) 2024-12-30 00:19:18 +01:00
Michael Vincent
f0db6a22d1 [WirecutterDealsBridge] Add bridge (#4359) 2024-12-12 17:52:41 +01:00
July
8234906127 [EpicGamesFreeBridge] Add new bridge (#4366) 2024-12-12 17:50:43 +01:00
July
d2370320e9 [ScribbleHubBridge] Get best-effort information during 403s (#4365) 2024-12-12 05:43:17 +01:00
July
9126b0f982 [CubariProxyBridge] Fix favicon properly (#4364) 2024-12-12 05:41:46 +01:00
Florent V.
4685bbdffd [EdfPricesBridge] fixing bridge (#4360)
* [EdfPricesBridge] add new brige

* [EdfPricesBridge] bad refactor

* [EdfPricesBridge] support php 7.4

* [EdfPrices Bridge] fix errors

---------

Co-authored-by: Florent VIOLLEAU <florent.violleau@samsic.fr>
2024-12-08 19:48:44 +01:00
Pavel Korytov
bf4a918e60 [MistralAIBridge] Add Mistral (#4356) 2024-12-05 17:30:21 +01:00
okbaydere
17d142c038 Add StorytelBridge for Storytel list fetching (#4355)
* Add StorytelBridge for fetching Storytel lists

* Updated StorytelBridge to include URL validation and cleaned up code
2024-12-04 18:54:24 +01:00
Predä
59d77d4576 [TikTokBridge] Include author profile picture (#4354) 2024-12-04 17:34:35 +01:00
Pavel Korytov
d956471d42 [QwenBlogBridge] Add bridge (#4353) 2024-12-02 16:46:13 +01:00
Dag
6a81fc0f51 fix(file_cache): if write failure, produce log record instead of exception (#4352) 2024-11-28 03:50:56 +01:00
July
88ccc6067c [CubariProxyBridge] Fix favicon (#4347) 2024-11-26 15:54:30 +01:00
Dawid Wróbel
c7f9870ba7 [OLXBridge] fix title and shiping info retrieval (#4346) 2024-11-26 03:04:02 +01:00
tillcash
c651e11b0f [MaalaimalarBridge] fix new url (#4344) 2024-11-25 19:03:35 +01:00
thomas-333
b42a993176 [Bluesky] New bridge (#4341)
* Create BlueskyProfileBridge.php

Bridge for Bluesky

* Update BlueskyProfileBridge.php

Attempt to fix test error

* Rename BlueskyProfileBridge.php to BlueskyBridge.php and add list of select data source

* Update BlueskyBridge.php to pass lint checks
2024-11-25 19:01:37 +01:00
SebLaus
ec6f98e3c2 Added Alternate way to get Price if no buttons available (#4342) 2024-11-24 18:11:57 +01:00
Sebastian Wolf
74496e23aa [MixologyBridge] add new bridge (#4331)
* [MixologyBridge] add new bridge

* [MixologyBridge] change invalid item property tags to categories

* [MixologyBridge] rewrite into FeedExpander

* [MixologyBridge] fix code formatting
2024-11-24 18:09:59 +01:00
User123698745
83bc3fd762 [DRKBlutspendeBridge] add new bridge (#4324)
* [DRKBlutspendeBridge] add new bridge

* [DRKBlutspendeBridge] move explode_lines into DRKBlutspendeBridge class
2024-11-24 03:57:28 +01:00
Dag
628b30208a fix: dont aquire exclusive locks (#4340)
Due to bugs in logging/error-handling there sometimes are deadlocks
2024-11-23 22:28:50 +01:00
Sebastian Wolf
e3260ff529 [NordbayernBridge] fill item categories if available (#4338) 2024-11-23 19:19:20 +01:00
Matt Connell
086ef7f8a7 feat: add WKYT bridge (#4337) 2024-11-23 19:12:36 +01:00
sysadminstory
2ee615e588 [PepperBridgeAbstract, DealabsBridge, HotUKDealsBridge, MydealsBridge] Streamlining Group Management (#4336)
* [PepperBridgeAbstract, DealabsBridge, HotUKDealsBridge, MydealsBridge] Streamlining Group Management

Since groups can change URLs, be created, or removed at the discretion
of website administrators, maintaining a valid and functional list of
groups is impractical.

Users can now enter the part of the URL that defines the group in a text
field, rather than searching through a lengthy, likely outdated list.

The way the RSS feed title is retrieved had to be adjusted accordingly.

Titles are now cached for 15 days to avoid unnecessary website access
and to prevent potential bot blocking.

Existing feeds will continue to work, as their parameters remain
unchanged; only the method for inputting them has been modified.

* [PepperBridgeAbstract, DealabsBridge, HotUKDealsBridge, MydealsBridge] Streamlining Group Management

Coding policy fixes

* [PepperBridgeAbstract, DealabsBridge, HotUKDealsBridge, MydealsBridge] Streamlining Group Management

Fix wrong comment

* [PepperBridgeAbstract, DealabsBridge, HotUKDealsBridge, MydealsBridge] Streamlining Group Management

Add Example values for Group context
2024-11-23 19:11:36 +01:00
Sebastian Wolf
a6e8760726 [FragDenStaatBridge] add new bridge (#4330) 2024-11-23 18:54:21 +01:00
July
9457e075f6 [PriviblurBridge] Fix invalid favicon, use either Tumblr or blog icon (#4327) 2024-11-23 18:50:40 +01:00
July
2294dac3f1 [AO3Bridge] Add fetch limit to reduce requests (#4328) 2024-11-23 18:47:08 +01:00
sysadminstory
6c86e2c1f7 [IdealoBridge] Really fix Logic and enhance Feed Content (#4321)
- Fix Feed Title generation (wrong usage of loadCacheValue)
- Use a more reliable way to get New and Used Price
- If no new Price and no Used Price are present in the page, then don't
  delete previous New Price and previous Used Price
- If there is no New Price and no Used Price, then return no Feed
  Item
- Fix the "now" date format
- Make the Feed Item Title more readable
- Use the Product Link as the Feed URL
2024-11-08 08:11:18 +01:00
Dennis
dd165ea9d1 [HuntShowdownNewsBridge] Fetches the latest articles from Hunt Showdown (#4318)
* feat: add Hunt Showdown News Bridge for fetching latest news articles

* chore: clean up formatting and remove unnecessary whitespace in HuntShowdownNewsBridge.php
2024-11-04 15:16:58 +01:00
Rose Liverman
1cd5b072f3 Formatting fix "For Hosts" documentation (#4317) 2024-11-03 18:33:05 +01:00
Tuğhan Belbek
8d6d0fa10c [DuvarOrgBridge] Add Duvar.org bridge for scraping news articles (#4315)
* Add Duvar.org bridge for scraping news articles

* PR Fixes

* Update DuvarOrgBridge.php to set a default value for the URL suffix

---------

Co-authored-by: Tughan Belbek <Tughan.Belbek@t-hive.io>
2024-11-03 18:30:28 +01:00
sysadminstory
bd0fb1da99 [IdealoBridge] Fix (#4316)
When a product was available before as used product in the past, and
now it's not available used anymore, a price update article was
generated on every feed loading, because the old used price was still
stored in the cache, and therefore different as "no price".

The issue was also present in the cas of a New product price that
becomes unavailable.

Now, when either there is no New or Used price available, the previous
price is delete from the cache.
2024-11-03 18:28:32 +01:00
Alexander Sulfrian
29d984cbe7 [TagesspiegelBridge] Add bridge for tagesspiegel.de (#4270)
* [TagesspiegelBridge] Add bridge for tagesspiegel.de

* [TagesspiegelBridge] Raise timtout to 60min
2024-11-03 18:25:51 +01:00
Arnav Jain
082542dabc [TestFaktaBridge] new bridge (#4307)
* [TestFaktaBridge] new bridge

* [TestFaktaBridge] fix linting errors
2024-11-03 18:22:44 +01:00
Arnav Jain
bc536f3928 [DäcksnackBridge] New Bridge (#4309)
* [DäcksnackBridge] new bridge

* [DäcksnackBridge] move preamble before figure
2024-11-03 18:20:48 +01:00
Matthieu Rakotojaona
c3dc46a307 [prtester] Update python dependency (#4311)
This is necessary for glob.glob() with the root_dir argument
2024-10-20 19:16:29 +02:00
User123698745
6c88f2c21e [prtester] fix prtester no longer supporting multiple bridges being changed, because the filenames are not unique (#4310) 2024-10-20 00:18:52 +02:00
Arnav Jain
668f3a9d7e [AppleMusicBridge] fix linting error (#4308) 2024-10-19 20:19:24 +02:00
somini
b9eb3c887a [PCGWNewsBridge] Remove bridge (#4305)
Fix #4291
2024-10-18 08:31:08 +02:00
Jonas Taedcke
f9a51b6768 [AppleMusicBridge] Further data request to receive artist information. (#4271) 2024-10-18 08:29:07 +02:00
Tuğhan Belbek
51cdb66f9c [HarvardBusinessReviewBridge] Add bridge (#4293) 2024-10-17 14:17:48 +02:00
Tostiman
bd88bc27d3 [TheDrive] New bridge (#4304) 2024-10-17 14:14:51 +02:00
Alexander Sulfrian
56994b3b5c [ZeitBridge] Remove content from original feed (#4260)
The original feed contains a small version of the header image and
the summary or a literal "None". The header image is already added, but
the original content was kept. This removes the original content and
adds the summary if it exists.
2024-10-17 08:47:44 +02:00
Bocki
664436c5f4 [prtester] Optimize tester workflow (#4303) 2024-10-17 01:25:07 +02:00
Bocki
70cf917f09 [ForensicArchitecture] Create ForensicArchitectureBridge.php (#4301) 2024-10-17 00:09:35 +02:00
Bocki
776e27218a [maint] fix phpunit test (#4300) 2024-10-17 00:00:52 +02:00
Bocki
e5e2059ed7 [maint] Update all workflow action versions (#4298) 2024-10-16 19:46:56 +02:00
Bocki
e7d6f89887 [ForensicArchitecture] Remove for bugfixing (#4297) 2024-10-16 19:21:24 +02:00
somini
0c96a47e8c Remove PanacheDigitalGamesBridge (#4277)
The Blog has a feed now:

https://panachedigitalgames.com/en/feed/
2024-10-16 19:14:06 +02:00
tillcash
5d83050673 [ForensicArchitectureBridge] Add Bridge (#4280) 2024-10-16 19:13:00 +02:00
vlnst
bd823100cd [maint] Update instance location (#4279) 2024-10-16 19:04:26 +02:00
Pavel Korytov
f89c75b4b8 [ArsTechnicaBridge] Fix the bridge after redesign (#4282) 2024-10-16 18:59:36 +02:00
Eugene Molotov
cdf21d48e5 [RutubeBridge] Multiple fixes (#4284) 2024-10-16 18:58:18 +02:00
Tostiman
3a5de759fa [CarThrottleBridge] update for new layout (#4285) 2024-10-16 18:57:44 +02:00
Tostiman
eb21e97d01 [OvertakeBridge] Renamed RaceDepartmentBridge to OvertakeBridge (#4294) 2024-10-16 18:37:30 +02:00
tillcash
6aba9fdf54 [MaalaimalarBridge] fix url (#4295) 2024-10-16 18:35:06 +02:00
Bocki
63c16e470d [prtester] Rework test storage (#4292)
* Update prtester.py

* Update prhtmlgenerator.yml
2024-10-16 15:36:57 +02:00
Mynacol
af26d845d9 Include all bridges in tarballs
Currently, two "demo" and "example" bridges are excluded from GitHub's
autogenerated tarballs. As I argued, those files can still be helpful
for integration tests, as they are run in NixOS and don't need internet
access or depend on the availability of external services [1].

Additionally, the official docker image builds from the checkout so it
includes those bridges when users use containers or a git checkout
compared to tarballs. This commit therefore unifies the list of
available bridges between deployment methods.

[1] https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/web-apps/rss-bridge.nix#L20
2024-10-09 18:10:52 +02:00
osvfj
80c43f10d8 [TCBScansBridge] Add bridge (#4263) 2024-09-12 11:07:22 +02:00
sysadminstory
d9316cdc60 [PicukiBridge] Try to fix the bridge (#4262)
This is a try to fix the bridge HTML parsing
2024-09-11 15:14:19 +02:00
tillcash
40041dd65f [DailythanthiBridge] fix url (#4261) 2024-09-09 19:06:08 +02:00
Pavel Korytov
358bebbb89 [EconomistWorldInBriefBridge] Fix bridge (#4258) 2024-09-07 05:02:27 +02:00
Dag
293d04f296 fix(spotify): detect rate limiting (#4253) 2024-09-03 07:02:37 +02:00
July
3dc8b65a0b [GovTrackBridge] Add feed for GovTrack events and blog (#4231)
* [GovTrackBridge] Add feed for GovTrack events and blog

* [GovTrackBridge] add missing default value

* [GovTrackBridge] leaner items array and limit implementation
2024-09-02 21:49:49 +02:00
Dag
486191b419 fix(cve_details) (#4251) 2024-09-02 21:43:40 +02:00
Dag
a6bdc322b0 refactor: extract exception and cache middleware (#4248) 2024-09-01 21:48:14 +02:00
bloominstrong
36fd72c87e [ABCNewsBridge] Fix broken due to site redesign (#4247) 2024-08-31 16:27:45 +02:00
Dag
9cabf60144 docs
* refactor

* docs
2024-08-30 04:37:40 +02:00
Dag
6a24e53d6c refactor (#4244) 2024-08-30 04:21:51 +02:00
Dag
bb2f471a03 fix: bug in prior fix (#4243)
Have to tweak the config BEFORE instantiating of course
2024-08-30 02:44:50 +02:00
Dag
3e1a8b29d9 fix: extract duplicate config loading (#4242)
Also fix a problem with bin/cache-prune and FileCache and its enable_purge option
2024-08-30 02:29:51 +02:00
Dag
9f48370eb0 fix: tweak caching logic (#4241) 2024-08-30 00:22:11 +02:00
Dag
39952c2d95 refactor: implement middleware chain (#4240)
* refactor: implement middleware chain

* refactor
2024-08-30 00:07:58 +02:00
Dag
e7ae06dcf0 fix: bug in prior refactor (#4239) 2024-08-29 23:02:01 +02:00
Dag
58544cd61a refactor: introduce DI container (#4238)
* refactor: introduce DI container

* add bin/test
2024-08-29 22:48:59 +02:00
tillcash
e010fd4d52 [HinduTamilBridge] fix image (#4237) 2024-08-28 19:45:54 +02:00
Petr Kolář
d51cc8f1a7 Fixed path in CeskaTelevizeBridge (#4236) 2024-08-28 19:43:40 +02:00
Dag
6516e31c1b refactor: format rendering (#4229) 2024-08-23 17:34:06 +02:00
Dag
c849576c93 fix(rumble): fix guid bug (#4232)
Remove tracking parameter in query to avoid feed readers to interpret these as new items
2024-08-23 17:09:17 +02:00
Clemens Neubauer
b0674d7b19 [BMDSystemhausBlogBridge] rework detectParameters (#4138)
* bridge BMDSystemhausBlog: rework of detectParameters

* fix lint phpcs error

* Update BMDSystemhausBlogBridge.php

* Update BMDSystemhausBlogBridge.php
2024-08-22 11:36:58 +02:00
Dag
05e2c350b7 refactor: less reliance on super globals (#4228) 2024-08-22 00:33:35 +02:00
July
4a3919c1a3 [NPRBridge] Add missing tag and remove extra HTML elements (#4227) 2024-08-21 23:05:29 +02:00
July
06a8896000 [PriviblurBridge] Add Priviblur (Tumblr frontend) bridge (#4221)
* [PriviblurBridge] Add Priviblur (Tumblr frontend) bridge

* [PriviblurBridge] prevent error if post has no tags
2024-08-21 22:58:26 +02:00
July
d379f3e575 [CubariProxyBridge] add bridge for cubari manga proxies (#4220)
* [CubariProxyBridge] add bridge for cubari manga proxies

* [CubariProxyBridge] add limit and use isset
2024-08-21 22:57:02 +02:00
July
3a327503ee [NPRBridge] add bridge for NPR stories (#4225)
* [NPRBridge] add bridge for NPR stories

* [NPRBridge] Use better selectors for multiple items
2024-08-21 22:10:03 +02:00
tillcash
2d5d2f5017 [NvidiaDriverBridge] fix typo (#4224) 2024-08-20 17:32:15 +02:00
tillcash
320afc3f32 [MaalaimalarBridge] fix image (#4222)
* [NvidiaDriverBridge] Added Windows support

* Update NvidiaDriverBridge.php

* Update NvidiaDriverBridge.php

* [MaalaimalarBridge] fix image

* [MaalaimalarBridge] fix lint
2024-08-19 19:17:42 +02:00
Dag
c0e37bcf35 refactor: frontpage and proxy setting (#4214) 2024-08-18 19:11:11 +02:00
Tobias Alexander Franke
e9d3a657ba [EASeedBridge] New bridge for the EA Seed blog (#4216)
* [EASeedBridge] New bridge for the EA Seed blog

* Fix linter issues
2024-08-15 00:47:39 +02:00
Tobias Alexander Franke
307c22204d [ActivisionResearchBridge] New bridge for the Activision Research blog (#4213)
* [ActivisionResearchBridge] New bridge for the Activision Research blog

* [ActivisionResearchBridge] Fix linting issues
2024-08-11 23:20:20 +02:00
Dag
4424ea54e9 chore: increase linter speed (#4211) 2024-08-11 02:31:50 +02:00
Dag
133dbf87c5 fix(telegram): add note if content is omitted from preview page (#4210)
* fix(telegram): add note if content is omitted from preview page

* lint
2024-08-11 01:23:10 +02:00
July
2e6e246759 [KemonoBridge] attempt to fix malformed tag responses (#4209) 2024-08-10 23:11:43 +02:00
Mynacol
129b8a3a5a [ModifyBridge] New bridge to modify feeds (#4164)
* [ModifyBridge] New bridge to modify feeds

Create a general bridge that can modify the common fields of feeds
with regular expressions.

* [ModifyBridge] Also modify <enclosure> element

Additionally to the list of <enclosures>.
2024-08-10 23:10:37 +02:00
July
4ef5ca50c6 [KemonoBridge] Add KemonoBridge (#4192)
* [KemonoBridge] Add KemonoBridge

* refactor

* [KemonoBridge] fix categories in cases where it's a proper json array

---------

Co-authored-by: Dag <me@dvikan.no>
2024-08-10 17:36:58 +02:00
Tone
adcc8e371d [TarnkappeBridge] changed "unwanted stuff" (#4206)
* [TarnkappeBridge] changed "unwanted stuff"

em was removed because the annoying affiliate info, but it also deleted the text from blockquotes.

The p-element with the affiliate info has no attributes like class, but it is the only p-element with a style-attribute, so I used this to identify it.

* Update TarnkappeBridge.php

removed whitespace

* Update TarnkappeBridge.php

don't know why I did it twice before
2024-08-09 15:20:10 +02:00
Dag
f358f1abec refactor: loadCacheValue/saveCacheValue (#4205) 2024-08-08 17:47:04 +02:00
Dag
2acd415475 refactor: drop usage of Debug::log (#4202)
* refactor: drop usage of Debug::log

* lint
2024-08-08 04:31:47 +02:00
Dag
6afd13eb06 refactor: deprecate FeedItem constructor (#4201)
* fix: bug in prior commit

* refactor: deprecate FeedItem constructor

* test: fix
2024-08-08 03:43:26 +02:00
Dag
2a96bf19b5 fix: bug in prior commit (#4200) 2024-08-08 02:55:35 +02:00
Dag
9973f731df feat: introduce RateLimitException (#4199) 2024-08-08 02:13:04 +02:00
tillcash
7073bb2f46 [NVIDIADriverBridge] Initial Commit (#4198)
* [NVIDIADriverBridge] Initial Commit

Fetch the latest NVIDIA Linux driver updates

* Update NVIDIADriverBridge.php

* refactor

* rename

---------

Co-authored-by: Dag <me@dvikan.no>
2024-08-08 01:35:48 +02:00
Quentin B.
db85015daa [AnfrBridge] Add bridge (#4191)
* [AnfrBridge] Add bridge

* yup

---------

Co-authored-by: Dag <me@dvikan.no>
2024-08-08 01:20:42 +02:00
Quentin B.
8c4385e61d [BodaccBridge] Add bridge (#4190)
* [BodaccBridge] Add bridge

* [BodaccBridge] Fix bridge

* [BodaccBridge] Fix API url

* fix

---------

Co-authored-by: Dag <me@dvikan.no>
2024-08-08 01:09:13 +02:00
Quentin B.
829d570f8e [CentreFranceBridge] Add bridge (#4189)
* [CentreFranceBridge] Add bridge

* [CentreFranceBridge] Fix bridge

* [CentreFranceBridge] Fix bridge

* [CentreFranceBridge] Improved icon choice

* [CentreFranceBridge] Fetch additional data from articles

* [CentreFranceBridge] New parameter to allow client to control how many articles to fetch

* [CentreFranceBridge] Improve bridge name based on existing parameters

* [CentreFranceBridge] Fixed some edge cases

* refactor: reorder

* fix

---------

Co-authored-by: Dag <me@dvikan.no>
2024-08-08 00:57:40 +02:00
Pavel Korytov
b25a779d98 [TldrTechBridge] Fix bridge (#4187)
* [TldrTechBridge] Fix bridge

* yup

---------

Co-authored-by: Dag <me@dvikan.no>
2024-08-08 00:27:33 +02:00
Christian Schabesberger
ee54cf4576 add NurembergerNachrichten bridge (#4185)
* add NurembergerNachrichten bridge

apply suggested changes and fix regions

put collectData on top

replace self:: with -> for methodcalls

* refactor: remove unused var

* refactor: order methods

* fix

---------

Co-authored-by: Dag <me@dvikan.no>
2024-08-08 00:00:26 +02:00
Dag
9215b95779 fix: bug in prior refactor (#4197) 2024-08-07 18:56:27 +02:00
Dag
c11bc184ca fix: restore php error_log writing (#4196) 2024-08-07 18:09:44 +02:00
Christian Schabesberger
313be4c512 replace self:: with -> for methodcalls in Nordbayern bridge (#4195) 2024-08-07 15:51:44 +02:00
Dag
4faaa79101 refactor: change the way dependencies are wired (#4194)
* refactor: change the way dependencies are setup

* lint
2024-08-07 03:15:43 +02:00
Dag
6ec9193546 yuop (#4193) 2024-08-07 00:21:06 +02:00
Eugene Molotov
401cc187b7 [RutubeBridge] Fix playlist mode returning empty result (#4184) 2024-08-02 17:44:46 +02:00
Dag
0051e0fcdd docs: improve docker docs (#4183)
* docs: improve docker docs

* fix: cleanup and remove duplicate docker instructions
2024-08-01 23:36:14 +02:00
Tone
d050fe9a9b [AnisearchBridge] fixed typo (#4182)
don't know why it was there
2024-08-01 12:36:26 +02:00
Dag
8ae716e75c fix: improve github issue template (#4181) 2024-07-31 21:57:33 +02:00
Pavel Korytov
b505667168 [SubstackBridge] Add Substack bridge (#4174)
* [SubstackBridge] Add Substack

* [SubstackBridge] Add docs

* [SubstackBridge] Fix lint

* [SubstackBridge] Update description

* [SubstackBridge] Update description (x2)
2024-07-31 21:57:20 +02:00
Dag
615c533587 fix(FeedParser): dont emit content module (#4180) 2024-07-31 20:34:33 +02:00
Dag
8a1f2604aa fix: bug in prior refactor (#4179)
* fix: bug in prior refactor

* fix deprecation notice
2024-07-31 19:25:51 +02:00
Dag
b8a9f34527 fix(FeedParser): scrape out content from rss content:encoded (#4178)
* fix(FeedParser): parse content module from rss2

* refactor
2024-07-31 19:04:07 +02:00
Dag
e55e9b8fac feat: enable all bridges by default (#4177) 2024-07-31 17:53:10 +02:00
Dag
9982bfce1f fix: convert php errors to exceptions when in debug mode (#4176) 2024-07-31 17:51:44 +02:00
Zack Puhl
1a8d0fb8ab [EBayBridge] fix undefined vars errors (#4175) 2024-07-31 17:51:05 +02:00
Dag
891c8979a3 refactor: return proper response object (#4169) 2024-07-31 17:30:06 +02:00
Pavel Korytov
aa3989873c [EconomistBridge] Add cookie (#4173)
* [EconomistBridge] Add cookie

* [EconomistBridge] Fix lint
2024-07-30 22:10:57 +02:00
MarKoeh
cb91afbd71 [ARDMediathekBridge] fixing API URL, start using show title (#4170) (#4172)
The bridge stopped working after the API server stopped accepting a trailing slash after the ID in the URL. This is being fixed. Also, the show title in the JSON was ignored. This is being fixed as well
2024-07-30 22:08:18 +02:00
Zack Puhl
22b39e3fcd [EBayBridge] Repair & Augment the eBay Feed (#4157)
* [EBayBridge]: discount details; fix DOM parsing

* [EBayBridge] Ending slash. No "www.ebay.commyhijack.net", for example.

* [EBayBridge] Trim discountLine details when set.

* [EBayBridge] Refactor and update content

* shameless self-addition to CONTRIBUTORS.md

* [EBayBridge] Toggle original search links w/ checkbox

* [EBayBridge] oops: fix introduced XSS vuln

* [EBayBridge] Fix linting error: use array_column

* [EBayBridge] fix compat with <php8
2024-07-29 17:53:39 +02:00
Zack Puhl
6d81d6d306 [RumbleBridge] Facelift, Validation, & Livestreams (#4160)
* [RumbleBridge] Facelift+media types (livestreams)

* [RumbleBridge] Remove 'required' from list input.

* [RumbleBridge] lint
2024-07-29 17:53:14 +02:00
Dag
955fb6f315 fix(reddit): increase default cache ttl (#4168) 2024-07-29 00:18:28 +02:00
Christian Schabesberger
8dd56bca05 fix bulletpoints for nordbayern (#4166) 2024-07-28 22:42:18 +02:00
Pavel Korytov
f773878459 [EconomistWorldInBriefBridge] Add cookie to options (#4165)
* [EconomistWorldInBriefBridge] Add cookie

* [EconomistWorldInBriefBridge] Add docs

* [EconomistWorldInBriefBridge] Best-effort to work without cookie
2024-07-28 22:41:08 +02:00
Eugene Molotov
d28a0fd94b [Vk2Bridge] Handling albums (#4163) 2024-07-28 22:34:12 +02:00
Eugene Molotov
bba225dfe8 [RutubeBridge] New option to fetch video from search results (#4162) 2024-07-28 22:33:48 +02:00
Tone
a1b3e596fc [AnisearchBridge.php] fixed youtube link (#4159)
$trailer->{'data-xsrc'} wasn't read correctly in EOT context
2024-07-28 22:21:14 +02:00
enwuenwu
2fcba49433 [Mailman2Bridge] fix message separation and improve "From_ lines" disambiguation (#4156)
* [Mailman2Bridge.php] enable PCRE_MULTILINE pattern modifier

Enable PCRE_MULTILINE pattern modifier on mbox content parsing. Without it parsing monthly archives results in only a single message each.

* [Mailman2Bridge.php] extend mbox "From_ lines" pattern

Extend PCRE pattern matching individual "From_ lines" used to split single messages in mbox content. 

In addition to the matching line having to start with 'From ' it now also has to end with time and date (hh:mm:ss yyyy). 

This makes the pattern slightly more robust against accidental matches when a line within the actual message body starts with 'From ' which Mailman 2 (Pipermail) may not be configured to disambiguate.

* [Mailman2Bridge.php] remove trailing slash from URI constant

---------

Co-authored-by: enwu <108224417+8279279374@users.noreply.github.com>
2024-07-28 22:11:48 +02:00
Tostiman
049af3cef7 [HardwareInfoBridge] delete bridge for discontinued website (#4124) 2024-07-28 22:03:35 +02:00
Dmitry R.
376e711f03 [NovayaGazetaEuropeBridge]: fix warnings (#4154) 2024-07-28 22:02:47 +02:00
tillcash
00d5242871 [GithubTrendingBridge] Add support for spoken languages (#4149)
* [GithubTrendingBridge] Add support for spoken languages

* Update GithubTrendingBridge.php
2024-07-28 22:00:36 +02:00
ORelio
f7ddbcd733 [GBAtemp] Fix title extraction (#4151)
Fix title extraction for news and reviews
2024-07-28 21:58:08 +02:00
tillcash
da8cfdf179 [HinduTamilBridge] refactor (#4146)
* [HinduTamilBridge] refactor

* [HinduTamilBridge] fixed lint

* [HinduTamilBridge] fixed lint 2

* Update HinduTamilBridge.php
2024-07-05 22:39:47 +02:00
Tone
4539eb69aa [GolemBridge] fix youtube links (#4144) 2024-07-04 20:53:49 +02:00
Niehztog
8bf1537054 delete obsolete bridge (#4143) 2024-07-04 20:53:16 +02:00
tillcash
d0c35146dd [HinduTamilBridge] Fix timestamp again (#4142) 2024-06-28 20:51:59 +02:00
Thomas
adad9d6405 [YouTubeCommunityTabBridge] Improve JSON extraction (#4140)
Small change that should make the extraction of JSON from HTML work more
reliably
2024-06-24 22:32:03 +02:00
July
2a84350cb2 [HumbleBundleBridge] Create new bridge (#4139)
* [HumbleBundleBridge] Create new bridge

* [HumbleBundleBridge] Use less redundant bundle type handling
2024-06-21 15:47:34 +02:00
Dag
d60f0b0e74 feat(FilterBridge): custom feed name parameter (#4136)
fix #4100
2024-06-18 21:12:29 +02:00
Dag
00074b9bfc fix: dont remove www from anchors in DOM, fix #4114 (#4135) 2024-06-18 20:55:05 +02:00
Dag
206bebc7bd ci: disallow the sizeof function in linter (#4134) 2024-06-18 20:22:46 +02:00
Mynacol
0eac7a0784 [HeiseBridge] Remove lost+found icon
Remove the icon visible in l+f articles, e.g.
https://www.heise.de/news/l-f-DISGOMOJI-die-Linux-Malware-die-auf-Emojis-steht-9765024.html

Using a css selector in the form img[alt*="l+f"] was tried, but is not
supported by the used library.
2024-06-16 13:23:36 +02:00
Ftonans
649dfa7292 Update instance list (#4131)
vern's instance seems to be working, I changed the url to https since they have automatic redirect.

I removed trailing slashes from the urls so they look the same.

I removed [rss.m3wz.su](https://rss.m3wz.su] since I didn't see the website online and the owner last posted on Fediverse two months ago. I'm not sure maybe it should be in "Inactive" category, I can try to contact m3wz for information about his instance.

I removed rss.foxhaven.cyou because of [this](https://shitpost.poridge.club/notes/9lumb2gll8) (TL;DR the owner lost access to the domain)

bus-hit is offline but the main website is working. I guess the rss-bridge just crashed and the owner will restart it.
2024-06-13 20:11:02 +02:00
sysadminstory
bb1e308057 [IdealoBridge] Fix price comparison and some PHP Notice (#4130)
* [IdealoBridge] Fix price comparison and some PHP Notice

- The prices were compared as String and the comparison was wrong in
  some case : now the price are converted to float before the
 comparison, so the logic works really.

- Don't show a new or used product price if it does not exist : this
  prevents a PHP Notice to be thrown

* [IdealoBridge] Fix price conversion in case the price is null

The conversion as float of the text price won't work if the price is
null : we retunr null in this case now.
2024-06-13 05:03:20 +02:00
July
e1b74aeb1b [GameBananaBridge] Add categories and more detailed updates (#4129)
* [GameBananaBridge] Add mod categorie(s)

* [GameBananaBridge] Include full update changelog details
2024-06-13 05:02:17 +02:00
tillcash
d3d33c72bd [HinduTamilBridge] fix timestamp (#4127) 2024-06-11 15:40:49 +02:00
Tone
87fa6ea71e [HeiseBridge.php] Prevent Youtube videos from being filtered out (#4125) 2024-06-10 19:40:07 +02:00
Tim-Florian Feulner
36706a3dec Fix NACSouthGermanyMediaLibraryBridge due to website changes (#4121) 2024-06-03 00:55:39 +02:00
tillcash
cfd406861e [HarvardHealthBlogBridge] Update (#4117)
Make article image optional as all images are representative
2024-05-30 16:08:08 +02:00
tillcash
bd90109c70 [HarvardHealthBlogBridge] New (#4116) 2024-05-29 21:16:10 +02:00
tillcash
5a68ee0c87 [HinduTamilBridge] New (#4115) 2024-05-26 17:21:14 +02:00
Albert Kiskorov
dc199ebf5c Fix: Ensure $time is set from innertext when datetime attribute is not found (#4111)
This commit addresses a bug where the $time variable is not set from the innertext of the $time_element when the datetime attribute is not found. The previous implementation only checked if $time was null or an empty string, which did not cover all cases where the datetime attribute might be missing. By using the empty() function, we ensure that $time is correctly set from the innertext when the datetime attribute is not present.
2024-05-19 14:37:59 +02:00
Mynacol
75f35391fa [HeiseBridge] Add missing <ol> elements (#4110)
The following article has <ol> elements that were missing.
Adding them to have the full content.

https://heise.de/-9714438
2024-05-18 16:51:00 +02:00
Mynacol
7bde7a56f9 [ZeitBridge] Fix linting 2024-05-18 16:35:24 +02:00
Mynacol
4d12aa2a9e [ZeitBridge] Remove annoyances, add content
Remove navigational elements, podcast images.
Add many more header images, article content in <ul> (and for ggod
measure in <ol>) and quotes with their content and not only their
author.

Extreme example:
https://www.zeit.de/campus/2024-05/protest-palaestina-universitaet-europa-uebersicht
2024-05-18 16:35:24 +02:00
Mynacol
a7ed3d56f9 [ZeitBridge] Prettify author field
By removing HTML tags (plaintext) and trimming it.
2024-05-18 16:35:24 +02:00
July
b785a4b64e ArsTechnicaBridge: restore categories lost by FeedExpander (#4030) 2024-05-17 21:29:17 +02:00
July
6e2aeda61d [GameBananaBridge] Include update contents in feed (#4103)
* [GameBananaBridge] Include update contents in feed

* [GameBananaBridge] Fix dynamic title property
2024-05-12 21:46:07 +02:00
July
4949900863 [ScribbleHubBridge] Handle 429 errors and use consistent GUID (#4104) 2024-05-12 21:45:14 +02:00
Alex Balgavy
776ee233bd [NOSBridge] fix bridge (#4102)
CSS selectors were no longer valid.
2024-05-12 20:30:23 +02:00
Facundo Tuesca
1c3024fca7 [MangaReaderBridge] Change feed title to manga name (#4092) 2024-05-08 00:25:45 +02:00
Patrick
d11b7f7754 Change URI for St. Johannes Blick (#4099)
Co-authored-by: Patrick <jummo@mailbox.org>
2024-05-05 23:30:38 +02:00
Eugene Molotov
f480209825 [YoutubeBridge] Fix empty result in search feed (#4098) 2024-05-05 23:30:23 +02:00
Thomas
d15960f955 [YouTubeCommunityTabBridge] Multi-image attachment support (#4091)
Adds support for multi-image attachments.
Also changes individual if-statments in "getAttachments" to if/elseif
as each post can apparently only have one attachment anyway.
2024-05-02 19:45:04 +02:00
Korytov Pavel
f3ca567159 [TldrTechBridge] Fix and improve bridge (#4090) 2024-04-27 10:35:59 +02:00
Thomas
d31f20758c [YouTubeCommunityTabBridge] Improve building of content & title (#4089)
* [YouTubeCommunityTabBridge] Improve building of content & title

Fixes truncated link hrefs in content and adds some general
improvements regarding the building of item content and item title

* [YouTubeCommunityTabBridge] Fix PHP deprecation warnings

Fixes the following deprecation warnings:

substr(): Passing null to parameter #1 ($string) of type string is
deprecated
2024-04-26 18:47:06 +02:00
Tone
154b8b9cdb Create TarnkappeBridge.php (#4085)
* Create TarnkappeBridge.php

* Update TarnkappeBridge.php
2024-04-19 19:08:58 +02:00
Mynacol
1f71d76ac1 [HeiseBridge] Remove additional ad banners
For example
https://www.heise.de/meinung/Kommentar-Microsofts-Sicherheitspraxis-wird-zur-Gefahr-und-das-BSI-schweigt-9686629.html
has two inline banners for a heise offering, not directly related to the
article. Removing all "inline" figures, which seems to catch all inline
unwanted elements, while avoiding removing useful figures/images.
2024-04-18 13:39:37 +02:00
sysadminstory
8c3e973b9f [PepperBridgeAbstract] Fix the "no result" detection (#4082)
The "no result" test did not work, it is fixed now.
2024-04-18 01:43:53 +02:00
llamasblade
97f5dafbc5 [HytaleBridge] Fix bridge not pulling all blog posts (#4079) 2024-04-16 17:58:05 +02:00
llamasblade
957a820931 [YandexZenBridge] Fix broken bridge for some channels (#4078)
Fixes #4071.

Major changes:
- the bridge's URI changed from zen.yandex.com to dzen.ru, as the former
  redirects to the latter (perhaps the bridge's name should be changed
  as well);
- the channel's URL is now required instead of the channel's username;
- two kinds of URLs are supported, one for channels with usernames and
  one for channels with IDs in their URL;
- the channel's real name, as shown in the webpage, is now used as the
  feed title.
2024-04-14 19:14:52 +02:00
Miika Launiainen
b4d397ff70 [YorushikaBridge] Fix getting date (#4077)
* Remove unnecessary variable

* Fix getting date
2024-04-14 19:13:31 +02:00
Arya K
89013faf7d Add Project Segfault Instance (#4076) 2024-04-13 15:59:25 +02:00
Korytov Pavel
428c6c3c66 [ScientificAmericanBridge] Update bridge (#4074)
* [ScientificAmericanBridge] Update bridge

* [ScientificAmericanBridge] Fix lint
2024-04-12 01:57:55 +02:00
Miika Launiainen
58c254ad3b [YorushikaBridge] Add language selection parameter (#4073)
* Add language selection parameter

* Fix typo

* Fix lint errors
2024-04-11 17:18:37 +02:00
Dag
a73b66f4d6 fix(ScientificAmericanBridge) (#4070) 2024-04-10 18:32:48 +02:00
sysadminstory
815dc180cc [PicukiBridge] Fix image URL (#4068)
Image URL does not need to be faked anymore, as the content/type is now valid.
2024-04-10 17:30:56 +02:00
July
7d6881732d [ScribbleHubBridge] Add list page feed creation (#4012)
* [ScribbleHubBridge] Add list page feed creation

* [ScribbleHubBridge] Add list title handling

* [ScribbleHubBridge] Don't include timestamp in List GUIDs

* [ScribbleHubBridge] Fix usage of dynamic property
2024-04-07 23:02:36 +02:00
Dag
4602f4f475 tweaks (#4065) 2024-04-06 18:07:45 +02:00
Mynacol
b3ac1d176c [FDroidRepoBridge] Simplify json retrieval (#4063)
* [FDroidRepoBridge] Simplify json retrieval

I looked into avoiding the writing-to-file and then reading-from-file altogether. Using a special file path that leaves the data in memory probably wouldn't work. But I'm unsure why we use the `index-v1.jar` file altogether.
The main F-Droid repo [lists](https://f-droid.org/en/docs/All_our_APIs/#the-repo-index) not only `index-v1.jar` (which only makes sense if we were to use the contained signature, which we don't), but also `index-v1.json` and `index-v2.json`. These json files can be fetched with `getContents`, optionally cached, and directly fed into `Json::decode` without using a temporary file. The HTTP transfer encoding can compress the file to a similar degree the jar (=zip) can. That's exactly what this commit uses.

Now the question is whether all the F-Droid repositories out there have this file. I went through the whole [list of known repositories](https://forum.f-droid.org/t/known-repositories/721) and only one repo misses the `index-v1.json` file: [Bromite](https://fdroid.bromite.org/fdroid/repo/index-v1.json). Under these circumstances we can depend on the availability of the `index-v1.json` file.

Closes #4062

* [FDroidRepoBridge] Cleanup not requiring Zip

With the last commit 1152386678, the zip
extension is not required anymore. Don't fail if it's not available.
2024-04-05 17:39:38 +02:00
Mynacol
d5aa3aef69 [FDroidRepoBridge] Fix example repo
The ttrss example/placeholder repo is offline, which fails CI jobs.
Replace it with a healthy repo and package to get working CI tests and comparisons.
2024-04-05 11:39:43 +02:00
sysadminstory
3ff2ef94e0 Fix docs : Replace relative links to files with full URL (#4059) 2024-04-04 19:28:56 +02:00
Dag
001dd47439 fix: small tweaks (#4057) 2024-04-04 19:12:04 +02:00
Dag
3cba984d22 fix(FDroidRepoBridge): unlink when json file is absent from archive (#4056) 2024-04-04 17:43:07 +02:00
sysadminstory
82606a479a [PepperBridgeAbstract,DealabsBridge,HotUKDealsBridge,MydealsBridge] Fix search URL, No results handling fixed, Thread title and Message URL handling (#4053)
* [PepperBridgeAbstract,DealabsBridge,HotUKDealsBridge,MydealsBridge] Fix search URL, No results handling fixed, Thread title and Message URL handling

Search URL has been updated according to the website.

If a search doesn't return any results, the HTML won't contain any
specific text now : the HTML structure is slightly different, so the
bridge has been updated.

The unnneded 'no-results' text is now removed from the specific bridges.

The board thread title has been removed from the content, so now we use
the page <title> element.

In case a board message is empty, there was an exception during the
filtering of message without URL.

* [PepperBridgeAbstract,DealabsBridge,HotUKDealsBridge,MydealsBridge] Fix search URL, No results handling fixed, Thread title and Message URL handling

Coding policy fixes
2024-04-04 04:08:29 +02:00
User123698745
94292af51b [prtester.py] fix url parameter encoding (#4052)
this will (at least) fix the pr preview of:
bridges/AnisearchBridge.php
bridges/BakaUpdatesMangaReleasesBridge.php
bridges/DesoutterBridge.php
bridges/IndiegogoBridge.php
2024-04-04 04:07:16 +02:00
Tone
f736da6fae [GolemBridge] fix for internal videos (#4051)
* [GolemBridge] fix for internal videos

with this internal golem-videos can be played directly from feed

* Update GolemBridge.php
2024-04-03 16:23:52 +02:00
Niehztog
fb66775ece [XPathAbstract] Refactor xpath abstract (#4047)
* refactor XPathAbstract, keep all functionality intact

* fix linter errors

* further simplify code

* set default value for raw item content to true, avoiding escaping of html tags in feed item contents by default
2024-04-02 23:14:25 +02:00
Dawid Wróbel
8f962383c2 [eBayBridge] fix Belgian eBay URL handling (#4050)
Fixes #3918
2024-04-02 01:01:23 +02:00
Dawid Wróbel
bb979e9e08 [AllegroBridge] fix logical condition on parameters (#4049) 2024-04-02 00:06:15 +02:00
Dawid Wróbel
a12bab9eed [AllegroBridge] ask for a complete cookie string, mere wcdx works no more (#4048) 2024-04-01 23:44:45 +02:00
Miika Launiainen
b4659786cb [GenshinImpactBridge] Small fixes (#4046)
* Switch json_decode to Json::decode

* Change regex delimeter from / to #

* Save item enclosures as list
2024-04-01 21:16:32 +02:00
July
7001fbaf49 [AO3Bridge] Fix bad heading selector (#4045) 2024-03-31 22:41:58 +02:00
Dag
d5d470cbc2 fix(dribble) (#4044) 2024-03-31 22:10:59 +02:00
Dag
182567e434 fix(bridges/DavesTrailerPageBridge): remove (#4043) 2024-03-31 21:52:53 +02:00
Dag
9682f74fc5 fix(cnet): author typo (#4042) 2024-03-31 21:37:51 +02:00
Dag
17a3b4c9d8 Fix 198 (#4041)
* fix(twitch): log instead of exception

* typo
2024-03-31 21:32:27 +02:00
Dag
73289324bd feat: add vendor http header to cached responses (#4040) 2024-03-31 21:02:55 +02:00
Dag
8ca1b90840 fix(NationalGeographicBridge) (#4039) 2024-03-31 20:07:14 +02:00
Niehztog
1c3c85d8ff [XPathBridge] Allow multiple categories (#4038)
* [XPathAbstract] allow multiple categories

* fix feed icons in two bridges

* fix warning

* fix linter errors
2024-03-31 18:46:07 +02:00
Miika Launiainen
d23fd2522c [GenshinImpactBridge] Fix bridge to use new API (#4011)
* [GenshinImpactBridge] Fix bridge to use new API

* Add category parameters back to not break existing feeds

* Fix lint error

* Remove whitespace
2024-03-31 03:46:23 +02:00
sysadminstory
b58d8b099b docs: Complete helper function documentation (#3911)
* docs: Complete helper function documentation

Complete documentation of the Helper functions

* docs: remove parameters and add a link to source

- Parameters removed
- Link to the file defining the function

* docs: fix links

Fix links to source files
2024-03-31 03:44:10 +02:00
Dag
545dc969d3 refactor (#4037) 2024-03-31 03:38:42 +02:00
Quentin de Longraye
24e429969f specify system section for enabling bridges (#4036) 2024-03-30 16:11:57 +01:00
Tone
e0be366258 Update AnisearchBridge.php (#4025)
* Update AnisearchBridge.php

added youtube trailer

* made trailers optional and reduced scraping to 5 articles if selected

* Update AnisearchBridge.php
2024-03-29 15:37:43 +01:00
sysadminstory
be445759b6 [PepperBridgeAbstract,DealabsBridge,HotUKDealsBridge,MydealsBridge] Move as much as possible to JSON (#4032)
As the website use more and more JSON, and JSON is a machine readable
format, I migrated as much as possible to the JSON.

This simplifies the Abstract class a lot, and the Bridge classes need
less language specifi strings.
2024-03-28 19:44:27 +01:00
July
db984d8a8b AO3Bridge: move tags to categories and remove duplicate fic summary (#4031)
* AO3Bridge: move tags to categories and remove duplicate fic summary

* [AO3Bridge] Fix tag html entity encoding
2024-03-28 19:43:17 +01:00
Tone
e251e358ff [HeiseBridge] fix for embedded youtube-videos (#4034)
* [HeiseBridge] fix for embbedded youtube-videos

with this the embedded youtube videos will work in the feed

* Update HeiseBridge.php

* Update HeiseBridge.php
2024-03-28 19:42:41 +01:00
Tone
0c2099a852 [GolemBridge] fixed embedded youtube videos (#4033)
* [GolemBridge] fixed embedded youtube videos

embedded youtube-videos can be played directly from feed now

* Update GolemBridge.php

* Update GolemBridge.php

* Update GolemBridge.php

* Update GolemBridge.php
2024-03-28 19:41:56 +01:00
Tone
fee5e269d0 Update CaschyBridge.php (#4027)
without removing the video-container-div the embedded youtube videos work again
2024-03-24 16:38:51 +01:00
Tone
2aace6c898 Added Bridge for Anisearch.de (#4023)
* Create AnisearchBridge.php

* Update AnisearchBridge.php

* Update AnisearchBridge.php
2024-03-22 21:01:16 +01:00
sysadminstory
3ed193eee2 [IdealoBridge] Update Bridge Meta data & (#4022)
The bridge meta data has been updated to reflect that the bridge works
for other international version of Idealo.

The Price trend is displayed on every price in the the Feed element
content. The same function is now used to show the price trend in the
Feed element title, to remove some duplicate code..
2024-03-22 09:44:42 +01:00
Patrick
58e2b56d40 Adjustment to new website layout (#4020) 2024-03-17 19:03:09 +01:00
Tone
a61524bf77 Update RedditBridge.php (#4019)
prevent error htmlspecialchars_decode(): Passing null to parameter #1
2024-03-17 19:02:51 +01:00
Tim-Florian Feulner
36147a082d Fix NACSouthGermanyMediaLibraryBridge for new website layout (#4014) 2024-03-15 19:20:04 +01:00
sysadminstory
e6cb5fdc89 [IdealoBridge] Fix Feed items & Feed title customisation (#4013)
- Feed items with new price tracking had "Max Price Used" instead of
  "Max Price New"
- Feed Title is now customised with the product name and the Price
  limits
- Fixed logic for saving prices in cache
- remove undefined variable notices
2024-03-13 23:47:46 +01:00
Dag
4bad1c140a fix(reddit): url encoding (#4010) 2024-03-12 23:59:10 +01:00
Dag
5b80af978f docs: improve README (#4009) 2024-03-12 19:46:21 +01:00
tillcash
ecf61f6fa7 [DailythanthiBridge] New Bridge (#4006) 2024-03-11 20:14:10 +01:00
Mynacol
254efc2812 [ZeitBridge] Remove doubled text
The first two paragraphs were repeated at the end of articles. The first
CSS selector filters those out (example 1).
The second CSS selector removes a "Zum Anschauen benötigen wir Ihre Zustimmung"
line from a poll widget. We can't load the widget successfully,
therefore we should remove all embeds that seem to use javascript
(example 2).

1: https://www.zeit.de/campus/2024-03/bundesregierung-wissenschaft-arbeitsvertrag-regeln
2: https://www.zeit.de/campus/2024-03/ausbildung-abgebrochen-gruende-azubi-aufruf
2024-03-10 22:27:32 +01:00
Jonathan Kay
84b93e0f8f [ComicsKingdomBridge] Fix/Rewrite of ComicsKingdom Bridge (#4003)
* Rewrite ComicsKingdom Bridge

Rewrite of bridge as the existing one no longer works:
- Now uses REST API
- Added optional limit to get desired number of comics
- Author now reflects the comic creators name
- Feed name and comic titles now pulled from site
- Added myself as the maintainer as I've been the one maintaining, and the existing code no longer is used

* Change API to URI to pass test

* Remove whitespace, add curly braces and switch to single quotes
2024-03-10 15:18:50 +01:00
tillcash
79699131e8 [MaalaimalarBridge] New Bridge (#4001) 2024-03-08 12:46:32 +01:00
July
f7c1b71939 NyaaTorrentsBridge: add torrent to enclosures and generate better feed name (#3996)
* NyaaTorrentsBridge: add torrent to enclosures and generate better feed name

* NyaaTorrentsBridge: fix accidental () in bridge name
2024-03-06 19:40:59 +01:00
July
7a7f8d5050 AnnasArchiveBridge: correctly handling partial matches and file links (#3997) 2024-03-06 01:28:24 +01:00
D5k H3h
683c968d64 [Rooster Teeth] Add Camp Camp channel (#3992) 2024-03-01 20:24:14 +01:00
Dag
4c355ba308 fix(FilterBridge): trim title so that regex filter works as expected (#3989)
The fix is in FeedParser, so this fixes all usages
of FeedParser where title is now trimmed.

fix #3985
2024-02-20 19:32:31 +01:00
xduugu
35f6e62e45 docker: Use pre-built curl-impersonate library from github releases (#3984)
The docker image is only available for `amd64` architecture and therefore
cannot be used for arm images.

Fixes #3983
2024-02-20 08:03:04 +01:00
hleskien
932f20d434 fixed date with time in LuftfahrtBundesAmtBridge (#3987) 2024-02-18 19:19:33 +01:00
Korytov Pavel
e65155f440 [OpenCVEBridge] Add bridge (#3978)
* [OpenCVEBridge] Add bridge

* [OpenCVEBridge] Fix tests

* [OpenCVEBridge] Fix description of the filter parameter
2024-02-16 22:24:13 +01:00
July
7813f4564e AO3Bridge: add options to fetch chapter contents and list titles (#3981)
* AO3Bridge: add options to fetch chapter contents and titles for list feeds

and add downloads for each fic to enclosures

* AO3Bridge: fix list default value

* AO3Bridge: fix erroneous dynamic property usage

* AO3Bridge: fix unit test failure for getURI
2024-02-16 04:14:17 +01:00
sysadminstory
4d15ffd2cf [PepperBridgeAbstract,DealabsBridge,HotUKDealsBridge,MydealsBridge] (#3982)
Exclude thread results

Some categories showed some thread in the middle of the deals : now only
the deals are handled

Updated the "no results" text to follow the sites changes
2024-02-16 03:58:15 +01:00
Dag
598ee5b51e fix(pinterest): set enclosure so it emits mrss media:content prop (#3980) 2024-02-14 16:02:54 +01:00
Eugene Molotov
257799be8e [Vk2Bridge] Alternative bridge for VK (#3878) 2024-02-10 15:59:39 +01:00
hleskien
8e8028b786 Adopt WebDriverAbstract as a solution for active (JavaScript) websites (#3971)
* first working version

---------

Co-authored-by: Dag <me@dvikan.no>
2024-02-10 04:42:22 +01:00
Dag
ff7840d60f chore: prepare for introduction of php-webdriver/webdriver (Selenium) (#3975) 2024-02-09 22:51:10 +01:00
Dag
df7b91a2a3 chore: upgrade composer root deps (#3974)
composer update --root-reqs
Loading composer repositories with package information
Updating dependencies
Lock file operations: 0 installs, 2 updates, 0 removals
  - Upgrading phpunit/phpunit (9.6.9 => 9.6.11)
  - Upgrading squizlabs/php_codesniffer (3.7.2 => 3.8.1)
Writing lock file
Installing dependencies from lock file (including require-dev)
Package operations: 0 installs, 2 updates, 0 removals
  - Upgrading phpunit/phpunit (9.6.9 => 9.6.11): Extracting archive
  - Upgrading squizlabs/php_codesniffer (3.7.2 => 3.8.1): Extracting archive
Generating autoload files
26 packages you are using are looking for funding.
Use the `composer fund` command to find out more!
No security vulnerability advisories found.
2024-02-09 22:39:45 +01:00
Dag
7b2ac36264 chore: move committed third-party deps to lib (#3973) 2024-02-09 22:27:35 +01:00
tillcash
46ac77590e [KilledbyMicrosoftBridge] Update: Adjusted content format for consistency (#3968) 2024-02-09 09:39:03 +01:00
Dag
6f731b20a9 fix(DarkReading): official rss endpoint changed (#3967) 2024-02-09 08:03:04 +01:00
Dag
8a6798a227 fix: escape token for html context (#3966) 2024-02-09 07:27:16 +01:00
Tone
ae2eb2f1d1 feat(Reddit): add parameter for web UI frontend 2024-02-08 20:05:24 +01:00
Korytov Pavel
cfef482366 [EconomistBridge] Handle 404s in feed gracefully (#3965) 2024-02-08 15:36:03 +01:00
Tone
75a0a779c0 Update HeiseBridge.php (#3963)
fix for broken article categories
2024-02-08 15:35:24 +01:00
tillcash
6bb04d48ed [KilledbyMicrosoftBridge] New Bridge (#3961) 2024-02-07 19:33:25 +01:00
Dag
6878eb26aa fix: changed dom (#3958) 2024-02-06 19:32:05 +01:00
sysadminstory
64f95b4990 [PepperBridgeAbstract,DealabsBridge,HotUKDealsBridge,MydealsBridge] Fix missing price, discount and ships from information (#3956)
- DealabsBridge
- HotUKDealsBridge
- MydealsBridge
Add the currency in the i8n data of the bridges

- PepperBridgeAbstract
The Price, discount data ans Ships from information are in the HTML
content anymore, so switched to the js-vue2 attributes
2024-02-06 02:23:12 +01:00
Scott Colby
66a6847fd0 Two fixes to DeutscheWelle (#3954)
* [DeutscheWelleBridge] Small URL fix.

Reset the $item's uri value after removing the tracking query string.

* [DeutscheWelleBridge] Fix "hero" images.

The main "hero" image for each article has src="" and relies on the
srcset attribute for the browser to pick the best image based on the
actual displayed size.

The call to `defaultLinkTo()` replaces the empty src with the article's
link, which, not being an image, breaks the image.

This change resets the src's of any such images back to "".
2024-02-06 02:21:30 +01:00
sysadminstory
7931f37a83 [PepperBridgeAbstract] Fix deal image scraping (#3953)
Deal Image was moved to a vuejs element, the deal image scraping was
fixed.
2024-02-05 23:30:18 +01:00
Tostiman
d175bab58e Fix car throttle bridge (#3925) 2024-02-04 18:28:12 +01:00
Dag
7c89712837 ci: fix broken docs build (#3948) 2024-02-03 13:56:56 +01:00
ljf (zamentur)
a14508d79b Add sans-nuage instance (#3947) 2024-02-03 12:58:36 +01:00
Dag
ca87562cab fix: prepare release (#3945) 2024-02-02 18:22:10 +01:00
Dag
b964dcd936 fix: early warning if file permission problem (#3944) 2024-02-02 18:06:08 +01:00
Clemens Neubauer
81be72ea04 Update BMDSystemhausBlogBridge.php (#3943) 2024-02-02 15:33:59 +01:00
Alexandre Alapetite
8423c52606 Fix Docker exec (#3941)
fix https://github.com/RSS-Bridge/rss-bridge/issues/3940
2024-02-01 16:57:14 +01:00
Dag
d01c462ad5 fix(FeedExpander): if parse fails, include offending url in exception message (#3938)
Also some refactors
2024-01-29 21:51:34 +01:00
Dag
b2c8475b2c fix(gates): json decoding (#3937) 2024-01-29 21:51:23 +01:00
Dag
c4fceab7b3 refactor(FeedParser): (#3928) 2024-01-29 21:51:06 +01:00
Dag
cfe3dcfe6d fix: remove twitter from readme (#3935) 2024-01-28 01:51:14 +01:00
tillcash
a15e578158 [FirefoxReleaseNotesBridge] Add New Bridge (#3930)
* [FirefoxReleaseNotesBridge] Add New Bridge

I'm uncertain about the reasons for the failed checks.

* Update FirefoxReleaseNotesBridge.php
2024-01-27 13:44:58 +01:00
Dag
c3a968193e fix: typo in previous commit (#3934) 2024-01-26 21:58:24 +01:00
User123698745
6938f06125 [prtester.py] Fix: Only select actual "option" elements of "select" elements to prevent "TypeError: argument of type 'NoneType' is not iterable" (#3932) 2024-01-26 21:45:46 +01:00
Dag
0e3a79fd78 fix: bug in cache-prune (#3933)
fixes Uncaught Exception: No cache type configured
2024-01-26 21:44:34 +01:00
Dag
e58c867a82 feat: token authentication (#3927) 2024-01-25 18:20:02 +01:00
Dag
d08d13f2c8 refactor: introduce http Request object (#3926) 2024-01-25 16:06:24 +01:00
Dag
9574c17ddc refactor/fix (#3924) 2024-01-25 13:03:00 +01:00
Dag
06b299e627 refactor: prepare for introduction of token based authentication (#3921) 2024-01-24 23:06:23 +01:00
hleskien
1262cc982c added new bridges (#3920)
* added new bridges

* lint

---------

Co-authored-by: Dag <me@dvikan.no>
2024-01-24 19:12:38 +01:00
Dag
487c692e68 fix: a few deprecation notices on php 8.2 (#3917)
* fix: a few deprecation notices on php 8.2

* tweak
2024-01-23 23:02:06 +01:00
Dag
4986119f1f fix(codeberg): semi-fix parsing of pull requests (#3916) 2024-01-23 22:25:43 +01:00
Dag
bd58266b80 fix(OLX): deprecation notice (#3915) 2024-01-23 22:05:03 +01:00
Dag
4973aaadf6 fix(spotify): deprecation notice (#3914)
8192: explode(): Passing null to parameter #2 ($string) of type string is deprecated in bridges/SpotifyBridge.php line 322
2024-01-23 21:45:22 +01:00
ORelio
feb2a686d7 [CssSelectorBridge] Move metadata retrieval to lib (#3913)
May become handy for making other bridges
2024-01-23 15:58:30 +01:00
ORelio
b6909942c8 [Releases3DS/Switch] Update URL (#3910)
Adjust domain name due to "www." variant having invalid TLS certificate.
2024-01-22 13:44:28 +01:00
Clemens Neubauer
6ac976b92e create bridge to BMDSystemhaus Blog (#3907)
* create bridge to BMDSystemhaus Blog

* Update BMDSystemhausBlogBridge.php

* Update BMDSystemhausBlogBridge.php
2024-01-21 16:26:25 +01:00
knrdl
72eea1bd3d [KleinanzeigenBridge] remove unnecessary params 2024-01-20 18:44:11 +01:00
knrdl
fffe4663cb [KleinanzeigenBridge] Add filter options 2024-01-20 18:44:11 +01:00
ORelio
a865b1073a [Next.Ink] Rewrite bridge (#3863)
Adjust to changed feeds and html template.
Not backward compatible with existing feeds.
2024-01-20 10:57:14 +01:00
ORelio
d960e0049d [NextINpact Bridge] Rename into NextInkBridge
Site migrated from https://www.nextinpact.com/ to https://next.ink/
2024-01-20 10:57:14 +01:00
ORelio
bb36eb9eb8 [CssSelectorBridge] Time/Thumbnail improvements (#3879) (#3901)
* Implement <time> metadata tag as timestamp source
* Add setting to include thumbnail as article header
2024-01-19 21:30:53 +01:00
ORelio
12a90e2074 Utils: Add Webp MIME type (#3900) 2024-01-19 21:30:06 +01:00
SebLaus
6408123330 [IdealoBridge] added Header with user-agent and fixed typo (#3897)
* Added header with useragent

* copy paste error from local test environment

* Fixed missing space in New before

* fixed missing space after comma in argument list
2024-01-19 03:59:47 +01:00
Dag
6eaf0eaa56 fix: add cache clearing tools (#3896)
Forgot to add these in #3867
2024-01-17 20:10:32 +01:00
Dag
191e5b0493 feat: add etag support to getContents (#3893) 2024-01-12 01:31:01 +01:00
July
d5175aebcc [ScribbleHubBridge] Get author feed title regardless of CloudFlare (#3892) 2024-01-11 20:09:45 +01:00
July
d9ac019550 [AnnasArchiveBridge] Add new bridge (#3888)
* [AnnasArchiveBridge] Add new bridge

* [AnnasArchiveBridge] Add missing exampleValue

* [AnnasArchiveBridge] Remove vestigial debug print
2024-01-11 00:42:57 +01:00
Dag
080e29365a feat(http-client): add http retry count to config (#3887) 2024-01-10 21:48:12 +01:00
ORelio
c7e8ddf486 CssSelectorComplexBridge: Use cookies everywhere (RSS-Bridge#3827) (#3886)
v2 after feedback from #3870
2024-01-10 21:47:34 +01:00
Dag
0eb4f6b267 fix(tiktok): remove duplicate leading slash in url path, fix #3884 (#3885) 2024-01-10 20:39:15 +01:00
Dag
491cb50219 docs: typo (#3883) 2024-01-10 00:25:36 +01:00
Dag
2e5d2a88f3 fix: only escape iframe,script and link for html output (#3882) 2024-01-09 21:36:42 +01:00
Dag
1fecc4cfc1 Revert "CssSelectorComplexBridge: Use cookies everywhere (#3827) (#3870)" (#3881)
This reverts commit 0c08f791ef.
2024-01-09 21:28:43 +01:00
ORelio
0c08f791ef CssSelectorComplexBridge: Use cookies everywhere (#3827) (#3870) 2024-01-09 20:34:56 +01:00
Dag
0bf5dbbc0b chore: add tools for manually administrating the configured cache (#3867) 2024-01-09 20:33:35 +01:00
Dag
3ce94409ab feat: support itunes namespace in top channel feed (#3776)
Also preserves other properties.
2024-01-09 20:18:33 +01:00
нездалисько
ea58c8d2bc Update 06_Public_Hosts.md (#3877) 2024-01-06 18:13:50 +01:00
sysadminstory
55ffac5bae [PepperBridgeAbstract, DealabsBridge, HotUKDealsBridge, MydealsBridge] (#3876)
Fix the Deal source link

The HTML does not contain the link to the "Deal source anymore", now only an
attribute does contain the information about the Deal Source.

The JSON data is now extraced for each Deal, and used to get the
Temperature and Deal Source.
2024-01-05 07:23:40 +01:00
Alexandre Alapetite
12395fcf2d Docker fix default fastcgi.logging (#3875)
Mistake from https://github.com/RSS-Bridge/rss-bridge/pull/3500
Wrong file extension: should have been `.ini` and not `.conf` otherwise it has no effect.
See https://github.com/docker-library/php/pull/1360 and https://github.com/docker-library/php/issues/878#issuecomment-938595965
2024-01-05 07:22:16 +01:00
Petr Kolář
0f6fa8034b Fixed selector in CeskaTelevizeBridge (#3872)
* Fixed selector in CeskaTelevizeBridge

* Fixed also description selector
2024-01-02 16:23:13 +01:00
Damien Calesse
e904de2dc9 [YGGTorrent] Update URI (#3871) 2024-01-02 16:22:39 +01:00
Dag
ef378663aa test: happy new year (#3873)
* test: happy new year

* yup
2024-01-02 16:21:52 +01:00
Dag
fac1f5cd88 refactor(reddit) (#3869)
* refactor

* yup

* fix also reporterre
2023-12-30 01:33:31 +01:00
Dag
7dbe106582 docs(nginx, phpfpm): improve install and config instructions (#3866) 2023-12-28 23:26:14 +01:00
Damien Calesse
2032ed18c4 [SensCritique] Update the content to add the image (#3865) 2023-12-28 19:51:15 +01:00
sysadminstory
f67d2eb88a [TikTokBridge] Use embed iframe to bypass scraping protection (#3864)
The Tiktok Website was totally changed using some "scraping" protection
(passing as parameter value generated somewhere in the bunch of
javascript to the "API URL" that was before). The iframe embed does not
have such protection. It has less information (no date, ...) but it's
better than nothing !
2023-12-28 13:53:06 +01:00
tillcash
5ab1924c4f Add WorldbankBridge and OglafBridge (#3862)
* Add WorldbankBridge and OglafBridge

* Update OglafBridge.php

Remove redundant parent call to parseItem and rename formal argument to improve code clarity.

* Update WorldbankBridge.php

fix lint
2023-12-28 13:50:34 +01:00
Damien Calesse
c8178e1fc4 [SensCritique] Fix bridge (#3860) 2023-12-27 13:17:49 +01:00
Florent V
ad2d4c7b1b [BridgeAbstract] use getParameters instead of static to allow overriding it from bridges (#3858) 2023-12-26 12:20:49 +01:00
Florent V
1938446385 [EdfPricesBridge] add new bridge (#3846)
* [EdfPricesBridge] add new brige

* [EdfPricesBridge] bad refactor

* [EdfPricesBridge] support php 7.4

---------

Co-authored-by: Florent VIOLLEAU <florent.violleau@samsic.fr>
2023-12-26 12:19:08 +01:00
sysadminstory
c9074facfe [GreatFonBridge] Remove bridge (#3857)
Website is unreliable, it's not useful to keep this bridge.
2023-12-26 12:18:42 +01:00
sysadminstory
9f163ab7c6 [FreeTelechargerBridge] Update to the new URL (#3856)
* [FreeTelechargerBridge] Update to the new URL

Website has changed URL and some design : this bridge is now adapted to
thoses changes

* [FreeTelechargerBridge] Fix example value

Example valuse seems to use an "old" template, switch to a newer example
that use the new template

* [FreeTelechargerBridge] Fix notice

Fix notice
2023-12-25 14:51:51 +01:00
xduugu
98dafb61ae [ARDAudiothekBridge] add duration to feed items (#3854) 2023-12-23 09:43:01 +01:00
July
ea2b4d7506 [ArsTechnicaBridge] Properly handle paged content (#3855)
* [ArsTechnicaBridge] Properly handle paged content

* [ArsTechnicaBridge] Remove normal site ad wrapper
2023-12-23 09:42:37 +01:00
Dag
f40f997405 fix: various small fixes (#3853) 2023-12-21 09:24:22 +01:00
Dag
4c5cf89725 fix(rumble): not all videos have a datetime (#3852) 2023-12-21 09:18:21 +01:00
Paul
a81acbe464 Merge pull request #3840 from Mynacol/golem-rm-pageheader
[GolemBridge] Remove multi-page page headers
2023-12-20 22:29:52 +01:00
Mynacol
4e40e032b0 Remove matrix reference
The main communications platform is still Libera.chat, matrix was only
provided by the hosted IRC-Matrix bridge. The bridge was turned off
already and won't come back.
2023-12-20 22:22:28 +01:00
Dag
98a94855dc feat: embed response in http exception (#3847) 2023-12-20 03:16:25 +01:00
Dag
0c6ffbf5a4 fix(gatesnotes): the unfucked their json (#3849) 2023-12-19 08:46:37 +01:00
Dag
3944ae68cb fix(reddit): use old.reddit.com instead of www.reddit.com (#3848) 2023-12-19 07:53:25 +01:00
Brendan Kidwell
b34fa2d278 RumbleBridge - new selector needed on user/channel page (#3843) 2023-12-17 17:08:40 +01:00
Mynacol
c5f586497f [GolemBridge] Remove multi-page page headers
On multi-page articles like [1], all the pages after the first one have
a page header that we add in the article content. When we tack the
pages together again, we don't need those extra page headers.

[1] https://www.golem.de/news/science-fiction-die-zehn-besten-filme-aus-den-spannenden-70ern-2312-179557.html
2023-12-16 11:21:19 +01:00
Paul
c9c2944e7c Merge pull request #3838 from Mynacol/golem-add-h2
[GolemBridge] Add h2 elements from article content
2023-12-16 10:49:29 +01:00
Mynacol
0116dde275 [GolemBridge] Add h2 elements from article content
Else some headers are just missing.
Example article with previously missing movie names:
https://www.golem.de/news/science-fiction-die-zehn-besten-filme-aus-den-spannenden-70ern-2312-179557.html
2023-12-16 10:43:27 +01:00
Tone
d4ae55733b Update GolemBridge.php (#3836)
deleted the code which adds the author to the feed, because the author is already in the original feed, so it is not needed.
2023-12-15 23:39:27 +01:00
ash
4e1fa946b4 add rb.ash.fail to list of public hosts (#3835) 2023-12-15 23:39:04 +01:00
Arnav Jain
d127bf6e00 [DagensNyheterDirektBridge] New bridge (#3834)
* [DagensNyheterDirektBridge] New bridge

* [DagensNyheterDirektBridge] Lint: Replace all tabs with space

* [DagensNyheterDirektBridge] Lint: Lines

Add empty lines and move start brace to new line

* [DagensNyheterDirektBridge] Lint: short- array syntax

* [DagensNyheterDirektBridge] Lint: short array syntax

Fix incorrect line ending

* [DagensNyheterDirektBridge] Lint: further lint fixes

* [DagensNyheterDirektBridge] Lint: final fixes
2023-12-15 23:36:50 +01:00
Dag
38e9c396cf fix(codeberg): css selector tweak (#3832)
* fix(codeberg): css selector tweak

* yup
2023-12-13 22:20:21 +01:00
Dag
0c4b498d4f fix(reddit): tweak internal cache logic (#3831) 2023-12-13 22:06:47 +01:00
Dag
d157816e07 fix(reddit): cache tweak for 403 forbidden (#3830) 2023-12-13 21:56:14 +01:00
Dag
f01729c86f fix(arstechnica): plus a few unrelated tweaks (#3829) 2023-12-13 21:40:13 +01:00
sysadminstory
0b67544f86 [PepperBridgeAbstract] Fix temperature handling (#3828)
Website has changed how the temperature is renderd : the bridge does
follow the new website structure
2023-12-13 21:09:48 +01:00
Guillaume Lacasa
a3b064f4ee Find PanneauPocket city id from page URL (#3825)
Co-authored-by: Guillaume Lacasa <git@adhess.net>
2023-12-11 17:38:39 +01:00
Raymond Berger
4a398a5b14 Update FacebookBridge.md - not working (#3823) 2023-12-09 11:52:57 +01:00
sysadminstory
3ef0226a08 [PepperBridgeAbstract] Fix Detection of "no deals found" and more (#3821)
- CSS styles showing there were no deals found has changed : CSS class
  was updated
- Relative Date handling : the minimum granularity of a relative date is
  the minute on the site. Seconds are therefore meaningless, and are now deleted.

MydealsBridge was missing one relateve date prefix : now every date is
parsed (I hope so !)
2023-12-08 06:25:39 +01:00
sysadminstory
c3d9383523 [FindfeedAction.php] Use relative URL in Feed Link (#3820)
The FindFeed action used absolute URL. This breaks the usage of RSS
Bridge behind a reverse proxy in a container.

Fixes #3801 for the Find Feed action
2023-12-08 06:24:43 +01:00
knrdl
deb9a7269e [MotatosBridge] add bridge (#3799)
* [MotatosBridge] add bridge

* [MotatosBridge] fix uid as string

* [MotatosBridge] add support for all regions

* [MotatosBridge] fix: region: "required" attribute not supported for list
2023-12-06 17:07:22 +01:00
Eugene Molotov
f3df283c4d [VkBridge] Fix single photo duplication (#3816) 2023-12-03 18:54:23 +01:00
Nick McCarthy
206edaedf5 [GoogleScholarBridge] Minor patch (#3814)
* Do not add RSS entry if Check for updates is found in the article title - avoids repeat entries
2023-12-01 22:36:26 +01:00
Niehztog
44ff2f2cf8 adds Super Mario Bros. Wonder to NintendoBridge (#3810) 2023-11-30 17:53:47 +01:00
Michael Bemmerl
ccc20849ff [SchweinfurtBuergerinformationenBridge] Don't include images with data URIs as enclosures. (#3811)
See also setEnclosures() in FeedItem.php: URIs with a path are required.
2023-11-30 17:52:51 +01:00
George Sokianos
609eed1791 KoFiBridge fix the "Call to a member function find() on null" line 39 (#3807) 2023-11-28 22:54:39 +01:00
Matt DeMoss
b037d1b4d1 [Threads] add bridge (#3805)
* initial working Threads bridge

* properly specify a default limit

* phpcs formatted
2023-11-21 17:00:02 +01:00
joaomqc
2b741b1c1b [SongkickBridge] add new bridge (#3803)
* [SongkickBridge] add new bridge

* [SongkickBridge] fix var reference and outdoor category

* [SongkickBridge] remove unnecessary string concat

* [SongkickBridge] fix if clause formatting

* [SongkickBridge] fix formatting and event title
2023-11-15 16:26:25 +01:00
knrdl
ef711cb30b [KleinanzeigenBridge] add new bridge (#3798)
* [KleinanzeigenBridge] add new bridge

* [KleinanzeigenBridge] fix missing timestamp

* [KleinanzeigenBridge] linting

* [KleinanzeigenBridge] fix end of list detection
2023-11-13 00:12:39 +01:00
knrdl
4919c53c10 [DemosBerlinBridge] add bridge (#3800) 2023-11-13 00:11:19 +01:00
Dag
b347a9268a feat: new bridge MangaReader (#3795) 2023-11-10 12:56:11 +01:00
SebLaus
e76b0601b3 [IdealoBridge] New Bridge to track prices on idealo.de (#3786)
* [IdealoBridge] Created

Checks the price of a given item on idealo.de. Can create an Alarm Message if a the price is lower than set or an Priceupdate if the price has changed.

* Changed Exec and syntax

* last fixes for remaining warning
2023-11-10 12:55:56 +01:00
sysadminstory
57b61c8787 [MydealsBridge] Fix keyword seatch (#3794)
When no result were found using the keyword search, some random deals
were displayed because the "not found" text has been modified : the
text is now up to date.

Some type in the textual name of the Bridge and texte about the website
name was fixed
2023-11-09 10:16:34 +01:00
wpdevelopment11
7a7fa876d2 [VkBridge] Fix regex that extracts page name (#3793)
Dot should be allowed in page names.
Precise rules for page names are available here:
https://vk.com/faq19715 (in Russian)
2023-11-08 16:40:24 +01:00
sysadminstory
a6310cff1a [GreatFonBridge] Add new Instagram Viewer Bridge (#3791)
Add a new Instagram Bridge not using Cloudflare DDoS Protection
2023-11-07 21:32:46 +01:00
sysadminstory
84b5ffcc7c [PepperBridgeAbstract] Fix Deal Origin and Shipping cost (#3790)
- Deal Origin was changed by the website : fixed the CSS class to get it
- Shipping cost had an extra SVG image in the content : removed the
  whole HTML tags from the content
2023-11-07 05:02:34 +01:00
Evgeny
8d0ddb579f Adding rss.m3wz.su instance to list of public hosts (#3789) 2023-11-03 17:42:34 +01:00
Niehztog
1dabd10e25 [NintendoBridge] Add new bridge (#3784)
* Adds new NintendoBridge

* fix item uids, fix feed title

* fix feed icon, adds item categories

* fix feed source uri

* make currentCatgory property nullable

* fix linter errors

* fix linter errors

* attempt to fix unit tests by assigning default category
2023-10-30 11:47:25 +01:00
ORelio
cee25d862d [html] clean data attributes (#3782)
Some feed readers had difficulties with attributes containing html tags
2023-10-24 19:57:25 +02:00
Ryan Stafford
d4e4c3e89a [FarsideNitterBridge] New twitter bridge (#3781)
* [FarsideNitterBridge] New twitter bridge

* example value

* lint fix
2023-10-23 23:12:05 +02:00
Park0
f134808a26 Marktplaats categories added (#3761)
* Update MarktplaatsBridge.php

* Update MarktplaatsBridge.php only main categories

As the whole list is too big only main categories are used for now.

* Renamed parameter 2 to sc

Renamed unused method to better reflect it usage

* Update MarktplaatsBridge.php Several fixed

Categories completed
Added a default empty one
Check if the input is not empty before using
Added helper methods to generate the categorylist

* Update MarktplaatsBridge.php

Set the methods to private for the CI
2023-10-22 17:36:36 +02:00
mruac
a6a4502209 [Itaku] extend the number of images shown in a post (#3780)
* minor fixes

- extended itaku post if post does not have all images

* phpcbf

* .

* resolve deprecated explode param

yay null coalesces
2023-10-21 11:54:50 +02:00
Manu
4722201281 Add one-click install to PikaPods (#3778) 2023-10-20 20:29:28 +02:00
ORelio
4f7451895b Fix: content.php: last-modified/if-unmodified-since (#3771) (#3772)
* Fix: content.php: last-modified/if-unmodified-since (#3771)

Fix exception if server sent invalid Last-Modified header
Add support for Unix time instead of standard date string
Send back standard RFC7231 date string instead of Unix time

* Fix: content.php: if-unmodified-since: cURL API

Use getTimestamp() as cURL expects that and will format the If-Modified-Since header appropriately.
2023-10-20 13:33:07 +02:00
ORelio
8ff39f64f7 [html] add data-orig-file tag (#3777)
Add support for data-orig-file tag in convertLazyLoading()
Remplace end() with array_key_last() as discussed in #3769
Fix typo in comment
2023-10-20 13:31:52 +02:00
Teemu Ikonen
658391263e Add 'itunes:duration' tag for items with duration (#3774)
* [{Atom,Mrss}Format] Allow itunes tags on items without enclosure

* [Arte7Bridge] Add $item['itunes']['duration'] value
2023-10-19 17:02:53 +02:00
ORelio
9056106c2d [CNet] Rewrite bridge (#3764) (#3770)
Bridge was broken.
Full bridge rewrite using Sitemap as source.
2023-10-18 19:13:33 +02:00
ORelio
7533ef12e3 [html] improve srcset attribute parsing (#3769)
Fix commas not being used for splitting, resulting in broken src URL in some cases:
srcset="url1.jpg, url2.jpg 2x" would give src="url1.jpg,"
2023-10-18 19:12:19 +02:00
ORelio
a41bb088f8 [CssSelectorBridge] Add more metadata tags (#3768)
Add og: variants for published/updated time and author
2023-10-18 19:10:52 +02:00
sysadminstory
8203196145 [ImgsedBridge] More robust data parsing (#3766)
Date Interval with the article "an" or "a" are now handled in a generic
way : every "article" is replaced by the number "1" instead of a
handling of multiple special case
2023-10-18 02:33:29 +02:00
Dag
563c2a345b refactor (#3763) 2023-10-16 03:43:18 +02:00
Dag
ef5bd83bd0 feat: preserve and reproduce podcast feeds (itunes rss module) (#3759) 2023-10-16 02:58:03 +02:00
Ololbu
408c2e5e91 [FicbookBridge] Fix timestamp (#3760)
Delete a year word after date digits: `DD m YYYY г., HH:MM` to `DD m YYYY, HH:MM`
2023-10-15 15:24:07 +02:00
Dag
f7f3ca0126 fix(tapas): bug in prior refactor (#3758) 2023-10-15 03:37:50 +02:00
Dag
611fabe46c fix(youtube): reduce excessive network calls (#3757) 2023-10-15 03:15:47 +02:00
Dag
2aa52aa99a fix(youtube): bug in prior refactor (#3756) 2023-10-15 01:13:17 +02:00
Dag
cf9558648e refactor: YoutubeBridge (#3755) 2023-10-15 00:08:18 +02:00
Dag
daef240cd2 test: add test for FeedParser (#3754) 2023-10-13 23:14:08 +02:00
Dag
5f37c72be0 fix(binance): plus some other tweaks (#3753) 2023-10-13 20:48:08 +02:00
ORelio
fd52b9b9a4 [CssSelectorFeedExpander] Fix ArgumentCountError (#3739) (#3751)
* [CssSelectorFeedExpander] Fix ArgumentCountError (#3739)

Fix ArgumentCountError (#3739) using new FeedParser class (#3740)
Implement default value for feed name / url if missing

* [CssSelectorFeedExpander] Skip empty fields in source feed

Fix empty feed properties being passed down from source feed
 rssbridge.DEBUG lib/FeedItem.php(177): Author must be a string!
 rssbridge.DEBUG lib/FeedItem.php(267): Unique id must be a string!

If "don't expand metadata" is checked, then source feed is passed
down verbatim (only content is expanded) so the debug messages
will persist, but the issue is in source feed, not in the bridge.
2023-10-13 19:27:33 +02:00
Dag
920d00480d fix(senscritique) (#3750) 2023-10-13 11:24:22 +02:00
Dag
49d9dafaec refactor: more feed parsing tweaks (#3748) 2023-10-13 02:31:09 +02:00
Dag
2880524dfc refactor: remove parent calls to parseItem (#3747) 2023-10-13 01:59:05 +02:00
Dag
e379019db2 refactor (#3746) 2023-10-13 01:02:19 +02:00
Dag
44fb2c98bc fix: various fixes (#3745) 2023-10-13 00:26:11 +02:00
Dag
382648fc22 refactor: FeedExpander::parseItem() descendants (#3744) 2023-10-13 00:25:34 +02:00
Dag
9bda9e246a refactor: FeedExpander (#3740)
* refactor: FeedExpander
2023-10-12 22:14:04 +02:00
Jisagi
6634291c67 NyaaTorrentsBridge - add max items again (#3743) 2023-10-12 21:24:08 +02:00
Dag
e55a88fb8e refactor(nyaa) (#3742) 2023-10-12 20:32:17 +02:00
Dag
6a72c56cdd fix: various fixes (#3741) 2023-10-12 19:49:04 +02:00
Dag
d21f8cebf6 fix(imgsed): parsing of datetime string (#3738)
* refactor

* fix(imgsed): parsing of date

date_interval_create_from_date_string(): Unknown or bad format (an hour) at position 0 (a)
2023-10-11 18:37:01 +02:00
Eugene Molotov
7e183915a9 [VkBridge] Fix missing feed title (#3737) 2023-10-11 18:28:54 +02:00
Eugene Molotov
145bd10f4c [VkBridge] Revert more universal regex for title generation (#3736)
In practice it lead to feed items to have "untitled".
Using previous regex with more covered cases.

Credits to https://t.me/votkot as author of regex
2023-10-11 18:16:57 +02:00
Dag
b6a9baff94 fix(cvedetails,tldrtech) (#3735) 2023-10-10 21:41:57 +02:00
ORelio
143f90da60 [WeLiveSecurity] Fix content extraction (#3734) 2023-10-10 19:34:16 +02:00
ORelio
47f52b5912 Add CSS Selector Feed Expander (#3732)
* Add CSS Selector Feed Expander

This bridge combines CssSelectorBridge with FeedExpander
Allows expanding a feed using CSS selectors

* Fix code linting

---------

Co-authored-by: ORelio <ORelio>
2023-10-09 08:48:21 +02:00
Park0
f97a3fa4d9 Fia.com document bridge (#3733)
* Create FiaBridge.php

F1 documents from fia.com

* Update FiaBridge.php

Fixed concat
2023-10-09 08:46:24 +02:00
Dag
5f777d4126 fix(codeberg): add temp fix (#3730)
they changed html for tag and commit
2023-10-05 15:36:35 +02:00
Niehztog
e376805249 [NiusBridge] fix parse error, fix image content-type (#3728) 2023-10-05 02:31:04 +02:00
sysadminstory
1cbe1a6f98 [PepperBridge] Fix date parsing (#3727)
Website changed the date display.
This fix adapt the date parsing to the new website date display
2023-10-03 23:15:10 +02:00
User123698745
59dd49671d [BridgeCard] add example value to info hint and allow using it by right click (#3726) 2023-10-02 03:02:57 +02:00
Dag
64582a64f1 fix(tpb): add category (#3725) 2023-10-01 21:19:27 +02:00
Dag
547af0d0d2 refactor: use Json::encode instead of json_encode (#3724) 2023-10-01 20:54:28 +02:00
User123698745
69da0dd583 [refactoring] replace direct use of curl with getContents (#3723)
+ some fixed warnings
2023-10-01 20:46:51 +02:00
Dag
41df17bc46 refactor (#3712)
* test: refactor test suite

* docs

* refactor

* yup

* docs
2023-10-01 19:23:30 +02:00
sysadminstory
0c92cf32d4 [ImgsedBridge] Fix and improvements (#3710)
* [ImgsedBridge] Fix and improvements

- Display an error if the user doesn't select at least an content type
  to display
- Unsplit the regular expression to make the URL of imgsed.com work too
- Remove the "hour part" of the publication date : the website shows
  only the number of days if the content is older than one day

* [ImgsedBridge] Fix and improvements

Fix syntax

* [ImgsedBridge] Fix and improvements

- Fix TEST_DETECT_PARAMETERS
- change detectParameters regular expression to match more instagram.com
  URLs

* [ImgsedBridge] Fix and improvements

- Fix date parsing for interval 'a day'

* lint

---------

Co-authored-by: Dag <me@dvikan.no>
2023-10-01 19:00:13 +02:00
Dag
7273a05f02 fix: google play and tiktok (#3722)
* fix(googleplay)

* fix(tiktok)
2023-10-01 18:53:50 +02:00
User123698745
d822d666c7 [prtester] improvements and fixes for prtester (#3721) 2023-09-30 22:09:59 +02:00
vdbhb59
6cf9dfb7c9 Updated public hosts page & updated welcome screen image to the latest (#3707)
* Updated inactive hosts & rearranged alphabetically (country) wise

1. Moved rb.vern.cc as the whole domain itself is down for sometime now.
2. Sorted all instances in country wise alphabetical order for better alignment.

* Uploaded latest welcome page snap

Uploaded a current version of welcome page, which also shows "find feed from URL" functionality.

* Added a public instance I stumbled upon, home-hosted in France

Added https://rss-bridge.cheredeprince.net/ which I found to be self-hosted at home somewhere in France.

* Reverted the sorting as requested in my PR #3707

Reverted the sorting as requested in my PR #3707
2023-09-30 15:06:48 +02:00
ORelio
3557e5ffd4 [CssSelector/Sitemap] Minor fixes (#3719)
- Apply title_cleanup to title from metadata (#3717)
- Metadata: Fix ld+json object/array confusion
- Sitemap: Also try /sitemap.xml well known url
2023-09-30 15:03:52 +02:00
Dag
2172df9fa2 fix: various notice fixes (#3718) 2023-09-29 19:17:03 +02:00
Dag
b9ec6a0eb4 feat: add manyvids bridge (#3716) 2023-09-29 00:39:24 +02:00
Dag
0de5180ded feat: improve sqlite cache robustness (#3715) 2023-09-28 22:21:56 +02:00
Dag
f9ec88fb45 ci: incease max line length to 180 (#3714) 2023-09-27 23:29:08 +02:00
User123698745
c04c0a5614 [core] prevent "*" in prtester whitelist, causing the script to generate a preview for every single bridge (#3713) 2023-09-27 00:07:46 +02:00
Dag
ae53adefad refactor: FeedItem::setTimestamp() (#3711) 2023-09-26 00:27:45 +02:00
Dag
f421c45b21 test: add feed item test (#3709)
* test: add feed item test

also some refactor

* yup

* yup
2023-09-25 22:32:15 +02:00
Dag
cd30c25b08 refactor (#3708) 2023-09-25 21:18:48 +02:00
ORelio
e1b911fc1f [CssSelectorBridge] Retrieve metadata for social media embeds (#3602, #3687) (#3706)
* [CssSelectorBridge] Metadata from social embed (#3602, #3687)

Implement the following metadata sources:
 - Facebook Open Graph
 - Twitter <meta> tags
 - Standard <meta> tags
 - JSON linked data (ld+json)

The following metadata is supported:
 - Canonical URL (may help removing garbage from URLs)
 - Article title
 - Truncated summary
 - Published/Updated timestamp
 - Enclosure/Thumbnail image
 - Author Name or Twitter handle

SitemapBridge will also automatically benefit from this commit.

* [php8backports] Add array_is_list()

Needed this function for ld+json implementation in CssSelectorBridge.

* [SitemapBridge] Add option to discard thumbnail

* [CssSelectorBridge] Fix linting issues
2023-09-24 23:07:43 +02:00
User123698745
09f3c1532a [core] improve pull request artifacts comment (#3705) 2023-09-24 21:13:01 +02:00
Dag
857e908929 chore: prepare 2023-09-24 release (#3703) 2023-09-24 20:53:07 +02:00
Dag
f321f000c1 feat: add url component (#3684)
* feat: add url library

* fix
2023-09-24 18:34:09 +02:00
Dag
437afd67e0 fix: various fixes (#3702)
* fix: symfonycasts

* various fixes
2023-09-24 18:15:14 +02:00
ORelio
ce353c1e4f [CssSelectorBridge] Fix URL filtering (#3676) (#3701)
Co-authored-by: tougaj <tougaj@users.noreply.github.com>
2023-09-24 16:12:30 +02:00
Dag
0dc6c66840 fix: add duration (#3699) 2023-09-24 00:03:21 +02:00
Dag
d33808ea9e fix: image (#3698) 2023-09-23 23:49:01 +02:00
Dag
0c69148cff fix(vice): news rss changed (#3694)
* fix: typo in prior commit

* fix(vice): news rss changed
2023-09-23 20:39:02 +02:00
Dag
bab02bf190 fix(flickr) (#3692) 2023-09-23 19:29:04 +02:00
Dag
f943f8d002 fix: typo in prior commit (#3693) 2023-09-23 19:28:52 +02:00
Dag
b3b0736761 feat: improve error/exception ui (#3690) 2023-09-23 18:54:14 +02:00
Dag
cb6c931b1f fix(duckduckgo): order by date (#3689) 2023-09-23 17:50:41 +02:00
Dag
07f49225d9 fix: bug in refactor (#3688) 2023-09-23 16:52:39 +02:00
Dag
a6a1d553d9 tweaks (#3686) 2023-09-22 20:59:45 +02:00
Dag
39d6710798 fix(twitch) (#3685) 2023-09-22 20:41:39 +02:00
mruac
a3c29f3a52 resolve comment (#3683)
https://github.com/RSS-Bridge/rss-bridge/pull/3617#issuecomment-1730244049
2023-09-22 09:38:05 +02:00
User123698745
7a9bfa1087 [YoutubeBridge] handle new youtube description system / fix missing description (#3682)
* [YoutubeBridge] handle new youtube description system

* [YoutubeBridge] fix unrelated warnings

* [YoutubeBridge] discard everything when one link can not be matched & add more boundary chars

* [YoutubeBridge] rebase on master & minor fixes
2023-09-22 05:40:13 +02:00
Dag
7329b83cc0 refactor: logger (#3678) 2023-09-21 22:05:55 +02:00
Julien Papasian
360f953be8 Fix #3674 - [Nautiljon] Remove requirement of cURL NSS (#3679)
Looks like it works with OpenSSL now.
2023-09-21 18:55:53 +02:00
Dag
0bf38e5c56 fix: small notice errors (#3677)
* fix notice

* fix notice

* tweak

* tweaks
2023-09-20 03:15:15 +02:00
Dag
e6aef73a02 refactor (#3668) 2023-09-20 02:45:48 +02:00
Scott Colby
cf7e3eea56 Add DeutscheWelle FeedExpander bridge. (#3673)
* [DeutscheWelle] Add DeutscheWelle FeedExpander bridge.

* [DeutscheWelle] Fix linting errors.
2023-09-15 23:41:08 +02:00
User123698745
3b91b1d260 [XPathBridge] add option to skip htmlspecialchars (#3672) 2023-09-15 01:58:06 +02:00
Dag
409236e48e fix: logic bug in 429 caching logic (#3669) 2023-09-14 03:26:01 +02:00
Dag
bb7f329e81 fix(instructables): migrate from dom to json api (#3667) 2023-09-13 22:48:37 +02:00
Alexandre Alapetite
0175e13712 Docker from Debian base image (#3500)
* Docker from Debian base image
* Fix expose https://github.com/RSS-Bridge/rss-bridge/discussions/3234
* Re-fix better logs https://github.com/RSS-Bridge/rss-bridge/pull/3333
* Update to Debian 12 Bookworm instead of Debian 10 Buster
* Use Debian packaging instead of having to keep track of and manually install -dev libraries, and with LTS support
* Update to PHP 8.2 instead of PHP 8.0

* Fix php.ini location

* Minor order changes
To optimise caching
2023-09-13 18:08:22 +02:00
sysadminstory
4323a11667 doc : clarification detectParameters function and test (#3666)
* doc : clarification detectParameters function

- clarification about the return value of the detectParameters function
- add info about the constant TEST_DETECT_PARAMETERS to allow automatest
  test of the function detectParameters

* doc : clarification detectParameters function

Add reference to the findFeed action to encourage the implenentation of
the detectParameters function

* doc : clarification detectParameters function

Add commas to improve reading

* doc : clarification detectParameters function

- fix link
2023-09-13 00:16:11 +02:00
mruac
4f5a492dde [BridgeAbstract] fix undefined index issue (#3665)
* .

* attempt to fix #2943
https://github.com/RSS-Bridge/rss-bridge/issues/2943

* Revert "."

This reverts commit c0b6ccfea6.

* lint

* Revert "attempt to fix #2943"

This reverts commit 9f1a66e48d.

* moved fix to BridgeAbstract

* fix undefined index

* lint
2023-09-11 13:18:00 +02:00
mruac
3e1e96e477 [PatreonBridge] resolve null coalescing issue (#3664)
* extend post presentation

* applied phpcbf

note: phpcs does not like long null coalescing chains

* resolved phpcs

* resolved github comment https://github.com/RSS-Bridge/rss-bridge/pull/3617/#issuecomment-1699568400

* .

* lint SteamAppNewsBridge
2023-09-11 13:15:14 +02:00
ImportTaste
a9cf1512e7 [SteamAppNewsBridge] Add tags filter (#3662)
Undocumented tags filter discovered through /ISteamWebAPIUtil/GetSupportedAPIList/v1/
e.g. /ISteamNews/GetNewsForApp/v2/?appid=1091500&tags=patchnotes
2023-09-11 03:35:09 +02:00
Dag
3178deb5a8 fix: mastodon, cache tweaks, docs (#3661)
* cache tweaks

* docs

* fix(mastodon): type bug
2023-09-10 23:35:40 +02:00
Dag
4b9f6f7e53 fix: rewrite and improve caching (#3594) 2023-09-10 21:50:15 +02:00
mruac
a786bbd4e0 DisplayAction: defaultchecked fix (#3654)
* .

* attempt to fix #2943
https://github.com/RSS-Bridge/rss-bridge/issues/2943

* Revert "."

This reverts commit c0b6ccfea6.

* lint

* Revert "attempt to fix #2943"

This reverts commit 9f1a66e48d.

* moved fix to BridgeAbstract
2023-09-10 04:15:05 +02:00
sysadminstory
078091752a doc : Add documentation for the Findfeed action (#3659)
* doc: Add documentation for the Findfeed action

Added the documentation to the Findfeed action

* doc: Add documentation for the Findfeed action

- Complete documentation
- fix typos
2023-09-10 04:03:38 +02:00
July
586d707ae4 [ArsTechnicaBridge] Add new bridge (#3657) 2023-09-09 09:19:09 +02:00
mruac
b3a7842448 [PixivBridge] Add cookie auth and options (#3653)
* added cookie mgmt and support for issue
https://github.com/RSS-Bridge/rss-bridge/issues/2759

* added image proxy option

* + mature and ai options, + cookie doc

* mention doc

* check cookie is auth'd
2023-09-06 16:16:25 +02:00
csisoap
dbe37cc302 [TwitterBridge] Filter out any promoted tweet (#3652)
* Filter out any advertise tweet

* Make some filter work, fix bug that may happen with tweet id list.

* clear phpcs warning, ignore line length warning
2023-09-06 16:14:11 +02:00
sysadminstory
52b90e0873 [Core] Fix Find Feed URL encoding (Really this time) (#3651)
- the URL was only partially encoded because encodeURI() was used
  instead of encodeURIComponent()
Now the whole URL is urlencoded, and the whole URL is passed as is in
the GET parameter 'url'
2023-09-06 03:46:47 +02:00
sysadminstory
38b957398a [AutoJMBridge] Fix content extraction (#3649)
* [AutoJMBridge] Fix content extraction

- Website changed, bridge was updated accordingly
- Added the function detectParameters
- Added the test array for the detectParameters function

* [AutoJMBridge] Fix test

Fix content of the TEST_DETECT_PARAMETERS array

* [AutoJMBridge] Update exaù^me value parameter

Example value was not valid anymore, so it was updated
2023-09-05 02:12:47 +02:00
sysadminstory
752098e0fa [Core] Fix Find Feed URL encoding (#3650)
The URL entered by the user was not URL encoded for the find feed
feature : this had lead to wrong content sent back to he server
2023-09-05 02:12:20 +02:00
User123698745
99b86c0e1c [GithubSearchBridge] repair bridge / handle new search ui (#3647) 2023-09-04 03:00:08 +02:00
mruac
b9fdd20f8f [FurAffinityBridge] added doc for #3638 (#3646)
* added custom cookie config

* appease phpunit

* added docs
2023-09-03 00:23:36 +02:00
Niehztog
92b2bc5e11 fixes extracting article images, article date/time and article author and item id (#3645) 2023-09-03 00:22:48 +02:00
R3dError
64000a2526 [NACSouthGermanyMediaLibraryBridge] Add new bridge (#3636)
* Init nac south bridge

* Rename bridge

* Refactoring

* Refactor

* Fix formatting

* Fix testing errors

* Change constants

* Update logo

* Remove author omission in descriptions

* Fix comment

* Add maintainer

* Rename bridge

* Add technical note to bridge description
2023-08-30 19:01:55 +02:00
csisoap
4d05d0beff [TwitterBridge] Add support for OAuth authorization. (#3628)
* Update TwitterClient.php

- Add OAuth authorization header.
- Add new endpoint.

* Update TwitterBridge.php

- Make some changes to support new endpoint.

* Update TwitterBridge.php

* clean up, fix warning

* fix warning

* fix warning

* remove oauth token

* fix wrong twitter id when encounter reply post.

* Update TwitterClient.php

* fix wrong twitter id cause by previous commit

* clear warning

* attempt to clear warning

* attempt to clear warning
2023-08-29 17:14:34 +02:00
mruac
9707586ee8 [PatreonBridge] Extend the presentation of parsed posts (#3617)
* extend post presentation

* applied phpcbf

note: phpcs does not like long null coalescing chains

* resolved phpcs
2023-08-29 17:09:05 +02:00
sysadminstory
52c59caf2f [Core] Find Feed : fix CSS for search results (#3643)
There was no margin at the bottom of the search result : in case of two
ore more results, the two blocks had no space between them.
2023-08-29 17:08:18 +02:00
mruac
f0ec797f4b [FurAffinityBridge] Option for instance host to add custom cookie (#3638)
* added custom cookie config

* appease phpunit
2023-08-29 17:05:37 +02:00
mruac
9e33a15b93 [FurAffinityBridge] Fix if search result contains hidden submission (#3637)
* Reverts to preview submission if full is hidden

* Reverts to preview submission if full is hidden

* revert

* added fallback to higher res preview if SWF

* amend
2023-08-29 17:04:10 +02:00
Paul
00a18a1cd1 Merge pull request #3639 from Mynacol/gitlab-epic
[GitlabIssueBridge] Add support for GitLab Epics
2023-08-27 13:32:04 +02:00
Mynacol
14607c07f6 [GitlabIssueBridge] Fix example values for MR
These values are used for testing and PR artifacts, but
https://gitlab.com/fdroid/fdroidclient currently has no MR !2099,
leading to a HTTP 404 error. This just uses issue #1 and MR !1.

To support epics, the specified repository is ignored.
2023-08-27 13:27:37 +02:00
Mynacol
999d5dce40 [HeiseBridge] Remove archive link for heise+
archive.ph is also not able to provide the full content of paywalled
heise+ articles.
2023-08-27 13:01:09 +02:00
Mynacol
c3b5b382ba [ZeitBridge] Remove broken paywall workaround
Clean up spoofing Google Bot as this workaround doesn't work anymore.
2023-08-27 12:57:36 +02:00
Mynacol
18a8a51271 [GitlabIssueBridge] Add support for GitLab Epics 2023-08-27 12:13:44 +02:00
t0stiman
0325c2414a fix carthrottlebridge (#3633) 2023-08-25 12:34:35 +02:00
Lars Stegman
eb4ff7099f CSS Selector Bridge 2 (#3626)
* [CssSelector2Bridge] Implement CSS Selector bridge 2

* [CssSelector2Bridge] Fix author not being loaded

* [CssSelector2Bridge] Remove unneeded time nullcheck

* Fix linting

* Fix failing test

* Implement PR fixes

* Update bridges/CssSelector2Bridge.php

Co-authored-by: ORelio <ORelio@users.noreply.github.com>

* Rename bridge and fix syntax error for php7

---------

Co-authored-by: ORelio <ORelio@users.noreply.github.com>
2023-08-22 21:28:16 +02:00
sysadminstory
7591b10219 [Core] New feature : User Interface to "Detect" Feed from an URL (#3436)
* [Core] New feature : User Interface to "Detect" Feed from an URL

Detect Action has been expanded to support returning a Feed in a JSON
format instead of a Redirect. Existing usage of the Detect action will
keep working as usual.

Frontpage template has now a section to display the Feed detection
result, and a button to start the Feed Detection.

A new JS file contains the necessary JS (Ajax and Event management) to
fill the Feed Detection section.

* Coding policy fixes

* [Core] New feature : User Interface to "Detect" Feed from an URL

- Switch from old school XMLHttpRequest to fetch
- Enhance UX of search results
- Revert to it's original content
- Switch to a new Action : FindfeedAction.php
- Switch to template literals instead of string concatenation
- FindFeed action could retrun multiple feeds
- Results are sent with an absolute URL
- Switch to Json::encode() helper function

* [Core] New feature : User Interface to "Detect" Feed from an URL

- Move specific JS code to rss-bridge.js
- Change HTML tag for the button to have a consistant style with th rest
  of the page

* [Core] New feature : User Interface to "Detect" Feed from an URL

- If no context is sent, assume there is only one unnamed context
- Find parameter name in global and currect context

* fix

* remove typo

---------

Co-authored-by: Dag <me@dvikan.no>
2023-08-22 20:44:36 +02:00
Dag
54045be951 fix(tpb): add missing cat (#3631) 2023-08-22 20:06:16 +02:00
Dag
3ac861a866 fix(twitch): Invalid argument supplied for foreach() at bridges/TwitchBridge.php line 115 (#3630) 2023-08-22 19:47:32 +02:00
veloute
c5cbab1231 Update TheGuardianBridge.php (#3629) 2023-08-22 17:59:06 +02:00
Eugene Molotov
959dd937b4 [VkBridge] Using more universal regular expression to generate item title (#3627) 2023-08-21 04:53:54 +02:00
John S Long
79e3f7f204 [MastodonBridge] Add support for excluding regular statuses (non-boosts/replies) (#3624)
* [MastodonBridge]: add support for excluding posts (non-boosts/replies)

* Update name of input

* Fix lint failures
2023-08-20 04:37:21 +02:00
Corentin Garcia
a1237d90f1 [RainbowSixSiegeBridge] fix links, date and img tag (#3619) 2023-08-15 16:16:06 +02:00
mruac
28077155ca [PatreonBridge] Resolve creator name in feed name (#3616)
* resolve creators without custom url

* hint how to enter creator with non-custom url

* applied phpcbf
2023-08-15 16:04:09 +02:00
mruac
7a1180c80f bridges: added Itaku.ee Bridge (#3615)
* Fix php8.2 deprecated warning

Fix php8.2 warning: `Deprecated: Creation of dynamic property is deprecated`

* modified tag presentation

* renamed to fit naming convention

* undo commit

* applied phpcbf and phpunit
2023-08-15 16:02:58 +02:00
Dag
ce72503df6 fix(codeberg): change dom selectors for timestamp and content, fix #3610 (#3611) 2023-08-11 19:02:38 +02:00
George Sokianos
d55994643d bridges: Added Ko-Fi.com bridge (#3609)
* Added Ko-Fi.com bridge

* Changed the exampleValue based on KoFiBridge-pr-context1 artifacts

* Fixed "Undefined array key 0" error

* fixed PHPCS issues

---------

Co-authored-by: George Sokianos <George.Sokianos@hostelworld.com>
2023-08-11 16:16:53 +02:00
Christian Schabesberger
11ea6aedfd hide dpa articles in Nordbayern News (#3608) 2023-08-10 23:59:37 +02:00
sysadminstory
52d3cce59d bridges: add context to detectParameters (#3607)
* bridges: add context to detectParameters

Some bridges did not return the context parameter but they used it in
the parameters

* bridges: add context to detectParameters

Fix test for InstagramBridge
2023-08-09 22:40:24 +02:00
ORelio
6cc4cf24dc [FuturaSciences] Fix content extraction (#3487, #3488) (#3606) 2023-08-09 20:10:15 +02:00
sysadminstory
1fcf67f14a [PepperBridgeAbstract] Fix deal origin (#3605)
Origin display has chenged : this commit follow the websites changes.

Fixes #3521
2023-08-09 17:36:02 +02:00
sysadminstory
f3896ed543 [ImgsedBridge] Add detectParameters feature to the bridge (#3604)
The bridge can detect the most common profile variation URL of
instagram.com or imgsed.com websites to extract the username.
2023-08-09 17:35:35 +02:00
ORelio
b86ee5778b [SitemapBridge] Add SitemapBridge (#3602)
* [SitemapBridge] Add SitemapBridge

This bridge is a variant of CssSelectorBridge.
Instead of retrieving article list from home page,
retrieves article list from SEO sitemap.xml.
Requires CssSelectorBridge to be installed.

* [SitemapBridge] Code linting
2023-08-08 15:02:01 +02:00
adminvulcano
43ec82179b [TldrTechBridge] Add Cybersecurity section (#3601) 2023-08-08 07:00:07 +02:00
Korytov Pavel
cf6d94dc2a [EconomistBridge] Fix strange image urls (#3600) 2023-08-08 06:58:08 +02:00
Niehztog
3e3481bd7a adds Nius bridge (#3599)
* adds Nius bridge

* fix linter errors

* fix linter errors

* fix linter errors

* fix extract author
2023-08-07 05:33:35 +02:00
User123698745
4976cd227e [FeedExpander] support xhtml content / content with child elements (#3598)
* [core] support xhtml content type in FeedExpander

* [FilterBridge] change defaultValue to exampleValue

* [core] support content with child elements in FeedExpander
2023-08-04 22:14:08 +02:00
Tone
d32419ffcf added the option for a sessioncookie in heiseBridge (#3596)
* added the option for a sessioncookie

with a valid cookie you can get full heise+ (paywall) articles

* formating

* lint

---------

Co-authored-by: Dag <me@dvikan.no>
2023-08-03 22:43:55 +02:00
User123698745
7661a78a43 [core] add bridge not found warning message to frontpage (#3591) 2023-08-03 03:10:24 +02:00
Dag
ed97ce8646 fix: dont fail for non-existing enabled bridge (#3589)
* fix: dont fail for non-existing enabled bridge

* yup
2023-08-01 19:35:15 +02:00
mruac
10f7b6f4f6 Fix php8.2 deprecated warning when using bridge specific configurations (#3587)
* Fix php8.2 deprecated warning

Fix php8.2 warning: `Deprecated: Creation of dynamic property is deprecated`

* fix

* refactor: remove unused method

---------

Co-authored-by: Dag <me@dvikan.no>
2023-08-01 19:35:06 +02:00
Dag
8e2353ad3e fix: write to cache only if data is was not cached, fix #3586 (#3588) 2023-08-01 06:19:42 +02:00
Dag
7e4807530e fix: various small fixes (#3580) 2023-07-31 20:43:18 +02:00
Dag
8b6eecea25 docs: add note about expensive operation (#3579) 2023-07-31 20:43:11 +02:00
ORelio
f8fd05f08f [CssSelectorBridge] Handling of missing links (#3585)
When using parent element as URL selector:

* If no <a> inside some elements, ignore them
* If no <a> inside ALL elements, report an error

Fixes #3573 #issuecomment-1656943318
2023-07-31 19:07:34 +02:00
User123698745
f957eea300 [FallGuysBridge] new bridge (#3584) 2023-07-31 01:05:38 +02:00
User123698745
93eecdf79f [core] fix new bridge PRs not generating html preview artifacts (#3583)
* [core] replace everything except bridge name to get a valid whitelist.txt

* [core] do not use hard code repository name to improve working with forks

* [core] trim bridge names from whitelist.txt to reduce chance of failure
2023-07-30 23:26:59 +02:00
mrtnvgr
3a57fc800b DoujinStyleBridge: Update html tags (#3581) 2023-07-30 06:46:16 +02:00
Dag
701fe3cfed fix: various small fixes (#3578) 2023-07-29 00:14:30 +02:00
Aaron F
11ce8b5dcd CVEDetails got a new HTML layout. (#3577)
This fixes the parser for CVEDetails.
2023-07-27 23:54:17 +02:00
Korytov Pavel
f5f76f111b [TldrTechBridge] Add Web Dev and Founders sections (#3576) 2023-07-26 22:59:49 +02:00
Korytov Pavel
bf4ea12719 [ScientificAmerican] Fix bridge (#3575) 2023-07-26 21:47:47 +02:00
ORelio
235c084820 [DilbertBridge] Remove bridge (#3574)
dilbert.com has closed down.
2023-07-26 20:41:48 +02:00
ORelio
977c0db382 [CssSelectorBridge] Improvements (#3537) (#3573)
* [CssSelectorBridge] Improvements (#3537)

* Improve parameter documentation / add tooltips
* Allow extracting content from home page instead of article page
* Keep titles from home page when every page <title> is the same

* [CssSelectorBridge] Code linting

* [CssSelectorBridge] Code linting (2)

* [CssSelectorBridge] Code linting (3)
2023-07-26 19:41:29 +02:00
csisoap
556bca58cf [TwitterBridge] Fix search, user, list ID (#3566)
* Add ability to fetch user, list tweet

* Fix user, search, list ID although list still broke

* clear whitespace

* Revert CACHE_TIMEOUT

* clear whitespace, change single quote

* Clear PHP warning, add ability to get full-text if truncated

* Clear PHP warning

* clear warning

* clear whitespace

* Add check condition for mediaDetails.

* Add whitespace

* Add try catch exception for get full-text tweet

* clear warning

* clear warning
2023-07-25 22:36:41 +02:00
Dawid Wróbel
2cc89b767c [AllegroBridge] fix non-functional bridge (#3571)
— fix cookie pattern
– use data analytics attributes wherever possible to avoid relying on obfuscated class names
— add support for promoted offers
— include sponsored and promoted offers by default
— some additional refactoring
2023-07-25 20:52:47 +02:00
Simon Alberny
1f6c2cd32c Allocine Sorties movie date added (#3569) 2023-07-24 16:25:09 +02:00
Dag
b6fab20601 docs: improve readme (#3560)
* docs: improve readme
2023-07-23 23:05:56 +02:00
Dag
74635fd752 fix(DisplayAction): improve error handling and cache logic (#3558)
* fix(DisplayAction): improve error handling and cache logic

* restore prev timeouts

* refactor

* yup

* test: fix unit test

* leave twitter client unchanged

* leave twitter bridge unchanged
2023-07-23 23:05:35 +02:00
Eugene Molotov
38ca124de0 [VkBridge] Better title generation (#3563)
1. Use first parargraph only
2. Remove tags
3. Allow to use comma and colon in title
2023-07-22 14:00:12 +02:00
Dag
39a8346c53 fix(pokemonnews): throw if antibot, #3327 (#3562) 2023-07-21 20:52:20 +02:00
Dag
d08b2616ef feat(twitter): use account icon as feed icon, fix #3348 (#3561) 2023-07-21 20:26:22 +02:00
Dag
0a118310cb fix(sqlitecache): store blob as blob (#3555)
serialize() can return output with null bytes and other
non-text data. The prior behavior truncated data
which later results in unserialize() errors.

This happens when e.g. caching an object with a private field
or when caching e.g. a JPEG file (starts with 0xFFD8FFE1)

Fixes errors such as e.g.:

unserialize(): Error at offset 20 of 24 bytes at caches/SQLiteCache.php line 51
2023-07-20 19:11:13 +02:00
Predä
663729cf19 [TikTokBridge] Use another way to get videos infos to include video link (#3557)
* [TikTokBridge] Use another way to get videos infos to include video link

* [TikTokBridge] Use cover if dynamicCover is empty

* [TikTokBridge] Add support for the rest of item params
2023-07-20 05:50:45 +02:00
Predä
2ffb54c7c2 [PicukiBridge] Add count parameter (#3556) 2023-07-20 00:52:09 +02:00
Dag
517c7f5c9b fix(cache): bug (#3554) 2023-07-19 22:18:42 +02:00
Dag
93620aa105 fix(cache): bug in cache logic (#3553)
It is possible to have a cached item with a very old mtime but it's technically expired.

So, check for presence of time and whether the time it is within 10 days
2023-07-19 22:05:26 +02:00
Dag
a4a328583a fix(reddit): set custom http ua to fix 429 errors (#3552)
* refactor

* refactor

* fix(reddit): set custom http ua to fix 429 errors

* lint
2023-07-19 06:39:17 +02:00
Dag
f91723d9e5 fix(memcached): do not flush entire cache, oops (#3551) 2023-07-19 05:18:26 +02:00
Dag
6254b8593e refactor(cache): extract and encapsulate cache expiration logic (#3547)
* refactor(cache): extract and encapsulate cache expiration logic

* fix: logic bug in getSimpleHTMLDOMCached

* fix: silly me, index should of course be on the key column

* silly me again, PRIMARY keys get index by default lol

* comment out the delete portion in loadData

* remove a few log statements

* tweak twitter cache timeout
2023-07-19 05:05:49 +02:00
sysadminstory
087e790ec1 [ImgsedBridge] Add new Instagram Bridge Alternative (#3550)
* [ImgsedBridge] Add new Instagram Bridge Alternative

Imgsed is a Website adverstised on instagram website, that's is not
behind Cloudflare Anti Bot feature.
You can select to display Posts, Tags, and Stories of a specific
username

* [ImgsedBridge] Fix empty defaultValue
2023-07-19 03:28:14 +02:00
mrtnvgr
4ce63c88aa Add DoujinStyleBridge (#3549)
* v1

* improve title

* search support

* random support

* fix categories

* add metadata to content

* fix linter errors

* i'm sorry
2023-07-18 20:48:29 +02:00
Paroleen
a1bae7a9a8 [SpotifyBridge] Add search API support (#3548) 2023-07-18 00:43:08 +02:00
Dag
08d16322e1 fix: bug in prior conflict merge (#3546) 2023-07-16 22:37:37 +02:00
Dag
440adf2f3b fix(githubissue): add 10 min cache (#3545) 2023-07-16 22:28:20 +02:00
Dag
a59793e8d6 refactor: extract CurlHttpClient (#3532)
* refactor: extract CurlHttpClient

* refactor

* interface
2023-07-16 22:07:34 +02:00
Dag
7b46b97abd refactor(spotify): replace manual curl with getContents (#3544) 2023-07-16 21:50:44 +02:00
Dag
310160fd92 feat: improve http 429 handling (#3541) 2023-07-16 07:18:38 +02:00
Dag
773eea196f fix(sqlitecache): bug in prior refactor (#3540)
fixes:

rssbridge.WARNING Trying to access array offset on value of type bool at caches/SQLiteCache.php line 72
2023-07-16 05:29:56 +02:00
Dag
e8420b9f39 fix(sqlitecache): log failed unserialization (#3539) 2023-07-15 22:44:26 +02:00
Dag
ef8181478d perf(SqliteCache): add index to updated (#3515)
* refactor(SqliteCache)

* perf(SqliteCache): add index to updated
2023-07-15 22:12:16 +02:00
Dag
eaea8e6640 fix: Undefined index: REMOTE_ADDR in non-web sapi' (#3538) 2023-07-15 21:16:23 +02:00
Thomas
e5729fdaac [YouTubeCommunityTabBridge] Fix PHP warnings for posts w/o text (#3536)
Because "contentText" is always present, PHP warnings were previously
generated for posts without text.
Existence of sub-property "runs" gets checked now to avoid this.
2023-07-15 15:18:09 +02:00
Ryan Stafford
c8039d483b [TraktBridge] new bridge (#3534) 2023-07-15 15:12:11 +02:00
Ryan Stafford
73d88dda46 [New Bridge] Strava bridge (#3533)
* [StravaBridge] new bridge

* [StravaBridge] gpx link, detect parameters
2023-07-15 06:11:00 +02:00
Ryan Stafford
ea0456ea08 [New Bridge] Qwantz bridge (#3531)
* [QwantzBridge] new bridge

* [QwantzBridge] title fix

* lint fixes
2023-07-15 05:58:57 +02:00
ORelio
9efdf24a6e Add CustomBridge (#3457)
* Add CustomBridge

For advanced users. Create RSS feed using HTML selectors.

* [CssSelectorBridge] Refactor, Allow Unexpanded

Rename bridge to CssSelectorBridge
Allow unexpanded feed, i.e. make feed from home page only (1 request)
Refactor bridge to put most of the code into protected functions
Makes the code more maintainable and allows inheritance for variants

* [CssSelectorBridge] Fix linting
2023-07-14 22:09:45 +02:00
alexvong243f
a234392f80 docs: improve discoverability of instagram related bridges (#3528) 2023-07-14 05:15:49 +02:00
Jisagi
b9102d7e87 Add Steam Group Announcements Bridge (#3527)
* Create SteamGroupAnnouncementsBridge.php

* Shorten implementation

Maybe this fixes the tests

* test

---------

Co-authored-by: Dag <me@dvikan.no>
2023-07-13 17:23:12 +02:00
Fake4d
0f2b55fbef Update Configuration.php - Old Version Date in new Release (#3526)
Old Version Date in new Release
2023-07-12 11:56:07 +02:00
Dag
69aa751f40 fix(cache): bug in prior refactor (#3525)
* fix(cache): bug in prior refactor

* yup
2023-07-11 23:26:22 +02:00
csisoap
b3bf95bfdd Replace token for Twitter. (#3522) 2023-07-11 23:17:38 +02:00
Dag
6c0e186d3f fix(jornaln): Array to string conversion at lib/BridgeAbstract.php li… (#3523)
* fix(jornaln): Array to string conversion at lib/BridgeAbstract.php line 320

* yup
2023-07-11 16:54:59 +02:00
Dag
c9a861e259 fix(cache): bug in prior refactor (#3520) 2023-07-09 15:24:29 +02:00
Dag
dfe78fb379 fix: various small fixes (#3519) 2023-07-09 10:08:30 +02:00
Dag
f0a504bb9a fix(FeedMerge): allow xml parse failure too (#3518) 2023-07-08 23:36:36 +02:00
Dag
7881c87bed fix: various small fixes (#3517)
* fix(asrocknews): Trying to get property src of non-object

Trying to get property 'src' of non-object at bridges/ASRockNewsBridge.php line 37

* refactor(http): tweak max redirs config

* fix(tiktok)

* fix(gizmodo)

* fix(craig)

* fix(nationalg)

* fix(roadandtrack)

* fix(etsy)
2023-07-08 23:21:55 +02:00
Dag
1a529fac46 fix(soundcloud): bug in prior cache refactor (#3516) 2023-07-08 22:53:23 +02:00
Dag
adc38e65d9 docs: add install method using composer (#3514) 2023-07-08 17:07:43 +02:00
Dag
0b95dc2d4f chore: synchronize composer.lock (#3513)
$ composer update
Loading composer repositories with package information
Updating dependencies
Info from https://repo.packagist.org: #StandWithUkraine
Lock file operations: 0 installs, 12 updates, 6 removals
  - Removing phpdocumentor/reflection-common (2.2.0)
  - Removing phpdocumentor/reflection-docblock (5.3.0)
  - Removing phpdocumentor/type-resolver (1.6.1)
  - Removing phpspec/prophecy (v1.15.0)
  - Removing symfony/polyfill-ctype (v1.25.0)
  - Removing webmozart/assert (1.10.0)
  - Upgrading doctrine/instantiator (1.4.1 => 1.5.0)
  - Upgrading myclabs/deep-copy (1.11.0 => 1.11.1)
  - Upgrading nikic/php-parser (v4.13.2 => v4.16.0)
  - Upgrading phpunit/php-code-coverage (9.2.15 => 9.2.26)
  - Upgrading phpunit/phpunit (9.5.20 => 9.6.9)
  - Upgrading sebastian/comparator (4.0.6 => 4.0.8)
  - Upgrading sebastian/diff (4.0.4 => 4.0.5)
  - Upgrading sebastian/environment (5.1.4 => 5.1.5)
  - Upgrading sebastian/exporter (4.0.4 => 4.0.5)
  - Upgrading sebastian/recursion-context (4.0.4 => 4.0.5)
  - Upgrading sebastian/type (3.0.0 => 3.2.1)
  - Upgrading squizlabs/php_codesniffer (3.6.2 => 3.7.2)
Writing lock file
Installing dependencies from lock file (including require-dev)
Package operations: 29 installs, 0 updates, 0 removals
  - Installing sebastian/version (3.0.2): Extracting archive
  - Installing sebastian/type (3.2.1): Extracting archive
  - Installing sebastian/resource-operations (3.0.3): Extracting archive
  - Installing sebastian/recursion-context (4.0.5): Extracting archive
  - Installing sebastian/object-reflector (2.0.4): Extracting archive
  - Installing sebastian/object-enumerator (4.0.4): Extracting archive
  - Installing sebastian/global-state (5.0.5): Extracting archive
  - Installing sebastian/exporter (4.0.5): Extracting archive
  - Installing sebastian/environment (5.1.5): Extracting archive
  - Installing sebastian/diff (4.0.5): Extracting archive
  - Installing sebastian/comparator (4.0.8): Extracting archive
  - Installing sebastian/code-unit (1.0.8): Extracting archive
  - Installing sebastian/cli-parser (1.0.1): Extracting archive
  - Installing phpunit/php-timer (5.0.3): Extracting archive
  - Installing phpunit/php-text-template (2.0.4): Extracting archive
  - Installing phpunit/php-invoker (3.1.1): Extracting archive
  - Installing phpunit/php-file-iterator (3.0.6): Extracting archive
  - Installing theseer/tokenizer (1.2.1): Extracting archive
  - Installing nikic/php-parser (v4.16.0): Extracting archive
  - Installing sebastian/lines-of-code (1.0.3): Extracting archive
  - Installing sebastian/complexity (2.0.2): Extracting archive
  - Installing sebastian/code-unit-reverse-lookup (2.0.3): Extracting archive
  - Installing phpunit/php-code-coverage (9.2.26): Extracting archive
  - Installing phar-io/version (3.2.1): Extracting archive
  - Installing phar-io/manifest (2.0.3): Extracting archive
  - Installing myclabs/deep-copy (1.11.1): Extracting archive
  - Installing doctrine/instantiator (1.5.0): Extracting archive
  - Installing phpunit/phpunit (9.6.9): Extracting archive
  - Installing squizlabs/php_codesniffer (3.7.2): Extracting archive
Generating autoload files
25 packages you are using are looking for funding.
Use the `composer fund` command to find out more!
2023-07-08 17:07:35 +02:00
Dag
61b307a9f9 fix(tiktok): feed item link (#3511)
* fix(tiktok): feed item link

* fix(tiktok): support entire url, for convenience
2023-07-08 17:07:20 +02:00
Dag
341649a8a4 fix: tweak the defaultly enabled bridges (#3510)
* fix: tweak the defaultly enabled bridges

enabled_bridges[] = FeedMerge
enabled_bridges[] = Filter
enabled_bridges[] = GettrBridge
enabled_bridges[] = MastodonBridge
enabled_bridges[] = Reddit
enabled_bridges[] = RumbleBridge
enabled_bridges[] = SoundcloudBridge
enabled_bridges[] = Telegram
enabled_bridges[] = ThePirateBay
enabled_bridges[] = Twitch
enabled_bridges[] = Twitter
enabled_bridges[] = Vk
enabled_bridges[] = XPathBridge
enabled_bridges[] = Youtube
enabled_bridges[] = YouTubeCommunityTabBridge

* add feed reducer too

* add tiktok too
2023-07-08 17:07:08 +02:00
Dag
91976f7d56 fix(file cache): acquire lock before writing (#3509) 2023-07-08 17:06:49 +02:00
Dag
8b996e3056 refactor: display action (#3508) 2023-07-08 17:06:33 +02:00
Dag
c1c8304fc0 refactor: dont create multiple instances of the cache (#3504) 2023-07-08 17:03:12 +02:00
Dag
b594ad2de3 fix: discard empty lines in whitelist.txt (#3507) 2023-07-07 11:25:36 +02:00
User123698745
ef0b86968c [PicnobBridge] fix missing images (#3506)
* [PicnobBridge] fix missing images

* [PicnobBridge] handle invalid relative date (e.g.: 'Just now')

* [PicnobBridge] fix code indent
2023-07-07 08:16:45 +02:00
somini
d49ea235f0 [JornalDeNoticiasBridge]: Remove bridge (#3505)
This is now broken, it was replaced by a React monstrosity. The API
doesn't even let me read the post data, it demands authentication.
Bummer.

Better remove it now, since it's worthless.
2023-07-07 08:14:31 +02:00
Dag
46f0e97c73 refactor: remove useless actual args to clearstatcache (#3503) 2023-07-06 19:39:40 +02:00
Dag
5e22459eb6 fix: remove unnecessary calls to purgeCache (#3502) 2023-07-06 18:52:19 +02:00
Dag
965d7d44c5 feat(sqlite cache): add config options (#3499)
* refactor: sqlite cache

* refactor

* feat: add config options to sqlite cache

* refactor
2023-07-06 15:59:38 +02:00
Dag
21c8d8775e fix: bug in #3442 (#3498)
* fix: bug in #3442

* docs: add inline doc on how to enable all bridges
2023-07-06 15:20:56 +02:00
Dag
caac7f572c refacor: improve cache interface (#3492)
* fix: proper typehint on setScope

* refactor: type hint setKey()

* typehint
2023-07-06 15:10:30 +02:00
Dag
f8801d8cb3 feat: add system config enable_maintenance_mode (#3497) 2023-07-06 15:09:44 +02:00
Dag
8f9147458d fix(gatesnotes): fix #3493 (#3494) 2023-07-06 03:36:01 +02:00
Dag
a9fd3b9e61 fix(CacheInterface): logic bug in getTime (#3491)
* fix(CacheInterface): logic bug in getTime

* test
2023-07-05 17:37:21 +02:00
Dag
18e1597361 fix(ph): consent cookie (#3490) 2023-07-05 17:06:23 +02:00
Patrick
e9af41d666 Add bridges for JohannesBlick Steinfeld, OM Online and UsesTech (#3489)
* Add bridges for JohannesBlick Steinfeld, OM Online and UsesTech

* Fixed linit alert
2023-07-05 16:43:59 +02:00
Dag
354317d010 fix(rumble): add timestamp to items (#3485) 2023-07-05 05:41:33 +02:00
Dag
84501cfc00 feat: add health check action (#3484) 2023-07-05 05:41:20 +02:00
Dag
82c22bd2b5 fix(feedmerge): allow a single feed to break, and dont break the whole bridge (#3476) 2023-07-05 05:41:01 +02:00
Dag
a21d496bc7 feat: add default arg to Configuration::getConfig (#3331) 2023-07-05 05:33:22 +02:00
Dag
cf920694d5 fix(futurasciences): fix broken bridge (#3488) 2023-07-04 23:34:24 +02:00
Thomas
bf0d771367 [YouTubeCommunityTabBridge] Fix getURI implementation
Previously the undefined property "feedUri" was accessed here, always
causing a fallback to the parent class
2023-07-04 17:03:30 +02:00
Dag
48385777b4 fix: php notices (#3482)
* fix(furaffinity): notice

* fox(releases3ds): remove references to non-existing vars
2023-07-03 10:48:33 +02:00
Dag
d8bc015efc fix: php notices (#3479)
* fix(jornaln): A non well formed numeric value encountered

fixes

A non well formed numeric value encountered at bridges/JornalNBridge.php line 89

* fix(reuters): fix notice
2023-07-03 00:39:01 +02:00
Eugene Molotov
0f14a0f6ee [YandexZenBridge] Add image to post (#3478) 2023-07-02 22:56:45 +02:00
Thomas
eb2b4747ae [YouTubeCommunityTabBridge] Add timestamps (#3477)
As YouTube doesn't provide precise dates for community posts, an
increasing multiple of 60 seconds is subtracted from each timestamp.
This ensures that the original order is always preserved, even if there
are multiple posts with the same date (e.g. "1 month ago").
2023-07-02 22:56:08 +02:00
Dag
bf73372d7f fix: dont be case-sensitive on env vars (#3475) 2023-07-02 06:47:21 +02:00
Dag
748fc9fd65 fix: various small notice fixes (#3474)
* fix(patreon): php notice

* fix(pepperbridge): php notice

* fix(ebay): php notice

* fix(tiktok): php notice

* fix(yandex): fix notice

* fix(justwatch): notice

* lint
2023-07-02 06:40:25 +02:00
Dag
372880b5ef fix: file cache tweaks (#3470)
* fix: improve file cache

* fix(filecache): log when unserialize fails
2023-06-30 22:31:19 +02:00
rmscoelho
cc91ee1e37 [JornalNBridge] getName() fix (#3456)
* [JornalNBridge] getName() fix

* [JornalNBridge] feed fixes
2023-06-30 15:54:53 +02:00
Arnav Jain
fece9ed344 [NotAlwaysBridge] Add new categories, remove duplicate header, and social meta (#3463)
* [NotAlwaysBridge] add new tags

* [NotAlwaysBridge] Remove duplicate header and social meta

* [NotAlwaysBridge] Add space to fix lint issues
2023-06-30 15:50:54 +02:00
Bocki
b6a263037a [core] Fix prtester for optgroups (#3467) 2023-06-30 15:41:00 +02:00
Mynacol
410ef85618 [GolemBridge] Strip <script> tags
Golem articles referencing their podcast contain a JavaScript reference
to Podigee. We don't want JS, so we strip it.

Example page: https://www.golem.de/news/podcast-besser-wissen-von-schlangenoel-und-sicherheit-2306-175185.html
2023-06-28 16:01:32 +02:00
Dag
8eabdbe5f8 fix(Twitter): properly find time line entries when ordering is inconsistent (#3461) 2023-06-27 16:11:41 +02:00
António Pereira
bd6f56383c [PresidenciaPTBridge]: Fix timestamp search (#3459) 2023-06-26 17:52:32 +02:00
ORelio
d4bc63ee98 [TheHackerNews] Update content extraction (#3458) 2023-06-25 19:01:57 +02:00
rmscoelho
1b02d4f49b [CorreioDaFeiraBridge] cache timeout + getName fixes (#3453)
* [CorreioDaFeiraBridge] cache timeout fix

* [CorreioDaFeiraBridge] cache timeout fix

* [CorreioDaFeiraBridge] getName() fix
2023-06-22 07:27:52 +02:00
rmscoelho
a4ed52ca30 [VideoCardzBridge] cache timeout fix + getName fixes (#3454)
* [VideoCardzBridge] cache timeout fix

* [VideoCardzBridge] getName() + title fix
2023-06-22 07:27:30 +02:00
rmscoelho
1769399da8 [ABolaBridge] cache timeout fix + getName fixes (#3455)
* [ABolaBridge] cache timeout fix

* [ABolaBridge] fix timestamp and image alt null

* [ABolaBridge] formatting fixes

* [ABolaBridge] getName() fix
2023-06-22 07:27:01 +02:00
rmscoelho
12ba6154f9 [JornalNBridge] cache timeout fix (#3452)
* [JornalNBridge] cache timeout fix

* [JornalNBridge] cache timeout fix
2023-06-21 11:15:36 +02:00
rmscoelho
8e35ebf482 [New Bridge] Jornal N (Portuguese local newspaper) (#3451)
* [New Bridge] Jornal N (Portuguese local newspaper)

* [JornalNBridge] formatting fixes
2023-06-21 05:17:11 +02:00
rmscoelho
61130e89b4 [ABolaBridge] timestamp (#3448)
* [ABolaBridge] timestamp

* [ABolaBridge] formatting fixes
2023-06-21 05:15:01 +02:00
rmscoelho
ebebb886c5 [ABolaBridge] feed fixes (#3446)
* [ABolaBridge] category name and url fixes

* [ABolaBridge] "Mercado" feed fix; feed name fix; img fix;

* [ABolaBridge] formatting fix
2023-06-20 17:26:55 +02:00
Matt Connell
6eaa31b999 [New Bridge] WYMT news bridge (#3444)
* feat: add WYMT bridge

* fix: phpcs error
2023-06-20 15:13:41 +02:00
rmscoelho
1d3888f22a [VideoCardzBridge] category name and url fixes (#3447)
* [VideoCardzBridge] category name and url fixes

* [VideoCardzBridge] error fixes

* [VideoCardzBridge] formatting fix

* [VideoCardzBridge] cache timeout removal
2023-06-20 15:01:41 +02:00
rmscoelho
60be4cdebd [CorreioDaFeiraBridge] adding timestamps; fixing categories; (#3445)
* [New Bridge] Correio da Feira (regional newspaper)

* [CorreioDaFeiraBridge] adding timestamp; fixing name

* [CorreioDaFeiraBridge] formatting fixes
2023-06-20 12:46:24 +02:00
rmscoelho
5a0bacbd8a [New Bridge] Videocardz.com bridge (#3442)
* [New Bridge] Videocardz.com Bridge

* [New Bridge] Videocardz.com Bridge

* [Videocardz.com] cache timeout increase

* [VideoCardzBridge] cache timeout change

* [VideoCardzBridge] formatting

* [VideoCardBridge] formatting fixes

* [VideoCardzBridge] formatting fixes
2023-06-20 12:45:50 +02:00
rmscoelho
0c808dc3a1 [New Bridge] Bridge for sports website A Bola (#3441)
* [New Bridge] Bridge for sports website A Bola

* [ABolaBridge] add thumbnail

* [ABolaBridge] formatting

* [ABolaBridge] formatting fixes

* [ABolaBridge] formatting fixes

* [ABolaBridge] formatting fixes
2023-06-20 12:45:34 +02:00
rmscoelho
98b72b2c5c [New Bridge] Correio da Feira (regional newspaper) (#3443) 2023-06-20 05:57:22 +02:00
Thomas
54d626d5cd [XPathAbstract] Use baseURI to fix relative links (if available) (#3439) 2023-06-17 17:53:00 +02:00
somini
1e470ef341 [PresidenciaPTBridge]: Fix title search (#3438)
This was changed on the site itself, in the last few days.
2023-06-17 06:13:09 +02:00
Dag
0a8fe57003 feat: enable bridges using env var (#3428)
* refactor: bridgefactory, add tests

* refactor: move defaultly enabled bridges to config

* refactor

* refactor

* feat: add support for enabling bridges with env var
2023-06-11 03:16:03 +02:00
Nick McCarthy
d9490c6518 GoogleScholarV2Bridge (#3415)
* Added google scholar v2 bridge with more functionality

* Corrected Sort By interpretation (this is weird on Googles part)

* Remove some debug statements

* Merged GoogleScholarBridge and GoogleScholarV2Bridge into GoogleScholarBridge with two contexts.

* Left V2 in Bridge Name

* Lint

* Update GoogleScholarBridge.php

* Update GoogleScholarBridge.php

* Lint.

* ;
2023-06-10 18:35:04 +02:00
Jisagi
eb799e59a6 [NyaaTorrentsBridge] Add custom fields (#3420)
* Update NyaaTorrentsBridge.php

* lint

* lint #2

* Sir Lint the Third

* Add torrent id to custom fields

* Proposed improvements
2023-06-10 18:28:00 +02:00
Eugene Molotov
ec1a3f4fe3 [YoutubeBridge] Unassign maintainer (#3431) 2023-06-10 18:27:49 +02:00
Eugene Molotov
80376830c5 [VkBridge] Handle some secondary attachments (#3430) 2023-06-10 18:27:32 +02:00
Shikiryu
e859497d6a [PicalaBridge] Fix article without image (#3429)
Co-authored-by: Clement Desmidt <clement@desmidt.fr>
2023-06-09 17:30:11 +02:00
Dag
ca351edbfe test: use correct path for bridges (#3427) 2023-06-08 23:44:26 +02:00
Dag
8f9eaae338 chore: fix ci (#3426) 2023-06-08 23:37:36 +02:00
Dag
fbaf26e8bf fix(html_format): add spacing below date if author is missing (#3425)
* small ui tweak

* remove unused <div>

* refactor: rename method

* refactor: inline const

* refactor
2023-06-08 23:04:16 +02:00
Simon Alberny
95071d0134 Add RemixAudioBridge (#3424) 2023-06-07 22:37:38 +02:00
Simon Alberny
3f8165207e Add RainLoopBridge (#3423) 2023-06-07 22:36:51 +02:00
Simon Alberny
08be0ad7a5 Add GoAccessBridge (#3422) 2023-06-07 22:36:21 +02:00
Simon Alberny
6fa1f349d9 Add AllocineFRSortiesBridge (#3421) 2023-06-07 22:35:54 +02:00
July
54957d2a03 [GameBananaBridge] Load all full quality screenshots (#3419)
Replaces the low quality preview images used previously
2023-06-06 20:00:55 +02:00
Tone
819e453064 remove newsletter ad from finanzflussBridge (#3417)
* remove newsletter ad

* whitespace
2023-06-02 20:28:29 +02:00
Dag
1636a84c25 fix(spotify): use non-predictable cache key (#3330)
* refactor

* fix(spotify): use non-predictable cache key
2023-06-02 20:22:28 +02:00
Dag
ee498eadf9 fix: move debug mode to config (#3324)
* fix: move debug mode to config

* fix: also move debug_whitelist to .ini config

* fix: move logic back to Debug class

* docs

* docs

* fix: disable debug mode by default

* fix: restore previous behavior for alerts

* fix: center-align alert text
2023-06-02 20:22:09 +02:00
Ryan Stafford
c5cd229445 [YoutubeBridge] Set icon (#3416) 2023-06-01 21:26:47 +02:00
July
845a8f7936 [MangaDexBridge] Add option to add chapter images to entries (#3412) 2023-05-28 18:23:01 +02:00
aysilu-kitsune
2f0784c287 added my instance (#3413) 2023-05-28 18:21:44 +02:00
piyushpaliwal
227c7b8968 Sleeper.com Alerts. Fixes #2234 (#3411)
* Sleeper.com Alerts. Fixes #2234

* fix: linter issue
2023-05-28 01:31:45 +02:00
July
01f731cfa4 [GameBananaBridge] Create new bridge (#3410) 2023-05-26 18:19:34 +02:00
mrnoname1000
87b9f2dd94 [core] Fix XPathAbstract while working around Simple HTML DOM bug (#3408) 2023-05-21 21:06:35 +02:00
Dag
f803ffa79a fix: ArgumentCountError: DOMDocument::getElementsByTagName() expects exactly 1 argument, 2 given, #3406 (#3407) 2023-05-21 19:59:39 +02:00
mrnoname1000
b5dbec4cc1 [AllSidesBridge] New bridge (#3405) 2023-05-20 04:15:56 +02:00
mrnoname1000
3e0d024888 [core] Fix defaultLinkTo for simple_html_dom objects (#3404) 2023-05-20 00:02:17 +02:00
Dag
cfe81ab2ac fix: Call to a member function setAttribute() on int, #3402 (#3403) 2023-05-19 16:05:52 +02:00
mrnoname1000
096c3bca73 [XPathAbstract] Fix relative links in fetched HTML (#3401)
* [core] Make defaultLinkTo compatible with DOMDocument

* [XPathAbstract] Fix relative links in fetched HTML
2023-05-18 13:50:50 +02:00
Tone
ecd717cf58 removing a-collapse (#3394)
it is only used for ads for their magazine
e.g.: https://www.heise.de/news/Eventtipps-fuer-Fotografen-und-Fotografiebegeisterte-9010049.html?seite=all
2023-05-12 23:41:08 +02:00
Tone
0c540b4637 added script to deleted elements in CaschyBridge (#3391)
* added script to deleted elements

Now it works much better with included content like twitter, e.g. in this article:
https://stadt-bremerhaven.de/1password-mit-android-14-wird-man-passkeys-in-chrome-und-apps-unterstuetzen/

* Update CaschyBridge.php

* Update CaschyBridge.php
2023-05-11 21:25:13 +02:00
mrnoname1000
d0f7f5e2d8 [New Bridge] FiderBridge (#3378)
* [core] Add config parameter to markdownToHtml

* [FiderBridge] New bridge
2023-05-11 21:24:12 +02:00
Alexandre Alapetite
e99e026fa8 Use standard Docker logs (#3333)
Instead of storing logs inside the container (where then cannot easily be seen not rotated), consider using the standard Docker approach of writing to standard output
https://docs.docker.com/config/containers/logging/
2023-05-11 01:44:11 +02:00
Dag
50865d5741 fix: add additional cloudflare title (Glassdoor specific) (#3342) 2023-05-11 01:33:38 +02:00
July
dc4134ed1d [ScribbleHubBridge] Add CloudFlare error handling (#3361)
* [ScribbleHubBridge] Set html defaultLinkTo

* [ScrubbleHubBridge] Add CloudFlare error handling
2023-05-11 01:33:21 +02:00
mrnoname1000
c6c4b3a24f [XPathAbstract] Fix encoding on feed title (#3365) 2023-05-11 01:32:01 +02:00
mrnoname1000
63dc500ae0 [XPathAbstract] Save HTML for entry content (#3366) 2023-05-11 01:31:34 +02:00
vincentvd1
723768c828 Add bridge for Magellantv articles (#3368)
* [MagellantvBrdige] added first version

* [MagellantvBridge]  cleanup, added tags and fixed bugs

* [MagellantvBridge] fix linting issues

* [MagellantvBridge] more linting fixes

* [MagellantvBridge] removed tabs
2023-05-11 01:30:25 +02:00
Tone
e7bda080b4 added iframe to $bad (#3380)
iframe can't be rendered in feed reader, so we can delete it
2023-05-10 22:14:34 +02:00
Joseph
8fd677f4ae [GithubTrendingBridge] Fix items (#3381) 2023-05-10 22:14:21 +02:00
Dag
88f646cf12 fix(TwitterBridge): trim screen name before passing it to twitter client (#3389) 2023-05-10 21:59:50 +02:00
Dag
49d105fd70 fix(TwitterBridge): remove ampersand from screen name, api dont like it (#3388) 2023-05-10 21:55:47 +02:00
Dag
ff49c9f731 fix(TwitterBridge): repair fetching of tweets by username (#3385)
* feat: alpha version of new twitter bridge

* fix: refetch guest_token if expired

* fix: purge cache

* fix: safeguards

* fix

* fix: two notices

* fix

* fix: use factory to create cache

* fix: fail properly instead of die()
2023-05-10 21:45:44 +02:00
Max
c628f99928 use lowercase 2023-05-10 18:40:33 +02:00
Tone
f26808d22c added article categories for CaschyBridge (#3379)
* added article categories

* whitespace
2023-05-08 16:21:39 +02:00
Tone
a1b6bca581 added article categories for GolemBridge (#3377)
* added article categories for GolemBridge

* tabs are bad, spaces good

* fixed duplicate categories on multi-page articles
2023-05-08 16:21:03 +02:00
Tone
ec091fb747 fixed authors and added categories for HeiseBridge (#3376) 2023-05-07 12:33:45 +02:00
mrnoname1000
887f4bbe15 [BugzillaBridge] Explicitly request JSON (#3364) 2023-04-27 19:24:29 +02:00
Paul Prechtel
212c56fde5 [HeiseBridge] Handle heise+ articles better (#3358)
- Stop parsing paywalled heise+ articles, as they had garbage content
  and anyways not the full article.
- Link to archive.today to access the full article without account.
  (Automatically getting the full article from archive.ph was not feasible
  b/c of captchas and problems extracting the actual content)
2023-04-20 23:02:08 +02:00
sysadminstory
00e716d84d [PepperBridgeAbstract] Fix "no results" check (#3357)
CSS class for "no results" text has changed, so the bridge has been
updated accordingly.
2023-04-20 11:22:53 +02:00
July
f0c96008bc [ScribbleHubBridge] Create new bridge (#3353)
* [ScribbleHubBridge] Create new bridge

* [ScribbleHubBridge] Improve 'Series' filtering

* [ScribbleHubBridge] Properly fetch feed name

* [ScribbleHubBridge] Fix feed name and set feed URI

* [ScribbleHubBridge] Fix linting violations with phpcbf

* [ScribbleHubBridge] Properly handle html encoding in titles
2023-04-19 20:35:04 +02:00
Eugene Molotov
343fd36671 [core] Remove hardcoded maximum duration of 24 hours in loadCacheValue (#3355) 2023-04-19 17:53:35 +02:00
Paul Prechtel
a4a7473abb [Docs] Fix link to SimpleHTMLDOM documentation (#3354) 2023-04-19 17:51:55 +02:00
Paul Prechtel
4068668de9 [ZeitBridge] Re-add paywall workaround (#3352)
Additionally to the Googlebot User-Agent, a Googlebot IP address has to
be used. For now, we can use `X-Forwarded-For` for this.
2023-04-18 18:41:40 +02:00
Eugene Molotov
7c4591c550 [VkBridge] Add detectParameters (#3351) 2023-04-18 18:41:11 +02:00
Paul Prechtel
0718fdc829 [ZeitBridge] Revert User-Agent (#3350)
The Googlebot User-Agent is no longer sufficient to circumvent the
paywall.
2023-04-17 15:33:14 +02:00
Dawid Wróbel
7eca527160 [eBayBridge] New bridge (#3349)
Fixes #3268
2023-04-15 18:40:49 +02:00
triatic
f1c54d5d55 [TwitterV2.md] Update to reflect price change to Twitter API (#3347) 2023-04-14 21:32:13 +02:00
Korytov Pavel
1ed7bdcddf [InternationalInstituteForStrategicStudiesBridge] Repair and improve bridge (#3338)
* [InternationalInstituteForStrategicStudiesBridge] Repair and improve bridge

* [InternationalInstituteForStrategicStudiesBridge] Fix lint
2023-04-08 22:09:07 +02:00
Paroleen
8486c0f8ca [SpotifyBridge] Add podcasts feed (#3329)
Co-authored-by: Matteo Parolin <matteoparolin99@gmail.com>
2023-03-24 20:34:51 +01:00
Eugene Molotov
249133204e [UI] Remove excessive top margin in all pages and logo in HTML feed preview (#3326) 2023-03-24 13:44:34 +01:00
Eugene Molotov
c8af9f9055 [VkBridge] Make timestamps more accurate (#3325) 2023-03-22 20:32:15 +01:00
Dag
9bb04ba848 Prepare 2023-03-21 release (#3323)
* fix: upgrade version string in php code
2023-03-22 19:32:19 +01:00
realansgar
307f5865c0 [ARDAudiothekBridge] fix feed icon not showing in RSS feeds (#3274)
* [ARDAudiothekBridge] fix feed icon not showing in RSS feeds

* [ARDAudiothekBridge] Fix lint errors
2023-03-21 18:24:28 +01:00
DRogueRonin
36e98e8481 docs: add docker development environment example (#3319) 2023-03-20 19:13:08 +01:00
Dag
347a0e9a3d fix: patch simple_html_dom, #3309 (#3310) 2023-03-20 19:12:13 +01:00
Dag
4c3ebb312d feat: improve error handling ux (#3298)
* feat: improve error handling ux

* feat: add error messages for failed xml parsing
2023-03-20 19:11:51 +01:00
Dag
9e9a697b8b feat: add config option "path" for file cache (#3297) 2023-03-20 19:10:01 +01:00
Miika Launiainen
4e616c7092 [YorushikaBridge] Replace YouTube embeds with YouTube link (#3321) 2023-03-19 12:50:04 +01:00
toineenzo
fbe7cc11ec Add more countries to App Store Bridge (#3246)
* Added more countries

* Fixed Brazil typo

* Update AppleAppStoreBridge.php

Removed whitespace line 52 for lint fix
2023-03-18 19:55:23 +01:00
sysadminstory
23fb5819cd [FreeTelechargerBridge] New bridge (#3318)
* [FreeTelechargerBridge] New bridge

New bridge

* [FreeTelechargerBridge ] Fix CACHE_TIMEOUT value

Fixed CACHE_TIMEOUT value
2023-03-16 09:35:49 +01:00
vdbhb59
dc8ce20482 Update 06_Public_Hosts.md (#3311)
Added my instance of public hosted RSS-Bridge host.
2023-03-14 15:55:14 +01:00
mad-reyk
224cce08a8 [Sfeed] Fixed category separator and random white spaces (#3308) 2023-03-12 15:21:21 +01:00
mad-reyk
c1f446fd19 [Sfeed] Added new format (#3306)
* [Sfeed] Added new format

* [Sfeed] Spaces instead of tabs

* [Sfeed] Move all global functions to class and fix phpcs warnings
2023-03-12 00:13:27 +01:00
Corentin Garcia
19fc2dc100 [GatesNotesBridge] Fix bridge (fix #3294) (#3305) 2023-03-11 23:26:22 +01:00
Dag
2c94791bcd fix: skip yt json if absent, fix #3301 (#3302) 2023-03-11 20:06:01 +01:00
Dag
1ffb2df46d New bridge (#3300)
Create rss feed from wallpapers published on erowall.com.

Allow fetching n latest wallpapers sorted by date, views, downloads and
tags.

Co-authored-by: Kurz Junge <kurz.junge.0xa@tutanota.com>
2023-03-11 01:41:02 +01:00
Miika Launiainen
dc9530b405 [YorushikaBridge] Created the bridge (#3299) 2023-03-09 18:36:51 +01:00
Bocki
90bf5518cb [Core] Activate live linting in codespaces (#3293)
* [Core] Add live linting to devcontainer

* Deactivate lint on type
2023-03-08 18:39:50 +01:00
Bocki
783160e715 [nginx] Add ipv6 listener (#3292) 2023-03-07 23:59:22 +01:00
Bocki
0a114c02c2 [Docu] Clarify docker instructions (#3291) 2023-03-07 23:58:21 +01:00
Bocki
2abdc7588a [Docu] Add documentation for Codespaces (#3289)
* [Docu] Add documentation for Codespaces

* Adapt PR line
2023-03-07 21:34:02 +01:00
Bocki
84e0135959 [Core] github codespaces setup (#3287) 2023-03-07 17:10:36 +01:00
Korytov Pavel
f7200756c3 [InternationalInstituteForStrategicStudiesBridge] Add bridge (#3286)
* [InternationalInstituteForStrategicStudiesBridge] Add bridge

* [InternationalInstituteForStrategicStudiesBridge] Fix lint errors
2023-03-07 17:03:50 +01:00
sysadminstory
b8ad49c562 [ExtremeDownload] Remove Bridge (#3285)
The Website has been taken down, this bridge is not needed anymore.
2023-03-07 01:02:51 +01:00
Dag
058e792b8f feat: add filecache config to enable/disable real purge (#3263)
* refactor: cachefactory

* feat: add filecache config to enable/disable real purge

* test: fix test
2023-03-06 21:50:40 +01:00
Dag
007f2b2d8a feat: sanitize root folder also in php error messages (#3262) 2023-03-06 21:47:25 +01:00
Dag
a01c1f6ab0 fix: disallow usage of default password (#3284) 2023-03-06 20:43:44 +01:00
Bocki
f0e5ef0fc5 [Various] getKey replacements and docu (#3283)
* [Various] getKey replacements and docu

* more bridges and fix to the abstract

* linting

* revert bandcampdaily. doing more than i thought
2023-03-06 20:01:51 +01:00
Tone
b40714079f Create FinanzflussBridge.php (#3282)
* Create finanzflussBridge.php

new bridge for finanzfluss.de

* Pascal case

* Rename finanzflussBridge.php to FinanzflussBridge.php

* Update FinanzflussBridge.php

more spaces!
2023-03-05 23:45:45 +01:00
Simon
180c332406 Update 06_Public_Hosts.md (#3280)
Change location for bridge.easter.fr
2023-03-05 00:37:42 +01:00
Joseph
8c4dbb32de [DockerHubBridge] Display compressed image size in items (#3279)
* [DockerHubBridge] Display compressed image size in items

* [DockerHubBridge] lint

* [DockerHubBridge] Use format_bytes()
2023-03-04 17:33:28 +01:00
Ololbu
5ab949ca55 [FicbookBridge] Fix new lines in content (#3278)
* [FicbookBridge] Fix new lines in content

Sets `$stripRN` in `getSimpleHTMLDOMCached` to `false` and replace new line to `br` through `str_replace()`.

* [FicbookBridge] Add space after comma
2023-03-04 16:12:46 +01:00
Bocki
f3f98a117c [Core] Add getKey function (#3275)
* [Core] Add getKey function
2023-03-02 13:25:57 +01:00
Bocki
f0d8cfd4d4 [JustWatchBridge] New bridge (#3273)
* [JustWatchBridge] New bridge
2023-03-01 20:24:01 +01:00
Korytov Pavel
4aed05c7b6 [TldrTechBridge] Add AI section (#3272) 2023-02-28 17:28:33 +01:00
Mynacol
4450e9b973 Let curl select the default HTTP version (#3249)
This essentially reverts b042412416,
as YouTube seems to have fixed their servers.
At least I was able to query the YouTube endpoint around 150 times with
CURL_HTTP_VERSION_2TLS recently. They even advertise HTTP/3 support with
an `alt-svc` HTTP header now.

This unsets CURLOPT_HTTP_VERSION to let curl decide
on the version. This would support all curl versions and opens the
possibility for HTTP/3, but leads to inconsistent behavior depending
on the underlying curl version.

We don't set CURL_HTTP_VERSION_NONE explicitly, as it is always the curl
default and opens the path to let individual bridges override the HTTP
version where necessary.

Alternatively, setting CURL_HTTP_VERSION_2TLS explicitly would lead to consistent behavior
regardless of the curl version, but might uncover old curl bugs before the
developers enabled HTTP/2 by default.
Additionally, that requires at least PHP 7.0.7 (we require PHP 7.4
already) and curl 7.47.0 [1], released on Jan 27 2016 [2].

See also the discussion on https://github.com/RSS-Bridge/rss-bridge/pull/3249

[1] https://www.php.net/manual/curl.constants.php
[2] https://curl.se/docs/releases.html
2023-02-22 17:48:39 +01:00
sysadminstory
7783e4133d [PicnobBridge] Update description & uid (#3267)
- Description now allows HTML and relative URLs are rewritten
- Post ID in the post URL is used as UID in the feed
2023-02-21 00:56:17 +01:00
Erisa A
4cfe69abb4 Update public instance (#3266) 2023-02-20 12:53:23 +01:00
Predä
db437b6326 [PicukiBridge] Add source field (#3265)
* [PicukiBridge] Add source_url field

* Simplify the regex matching.

* Add picuki source for feeds
2023-02-19 01:35:28 +01:00
Tone
5d4247dded Update CaschyBridge.php (#3261)
prevents the bug #3259 that quote-blocks will be displayed multiple times
2023-02-15 22:20:22 +01:00
Dag
286790727b fix: throw exception when outbox not found, #3255 (#3260) 2023-02-15 21:42:05 +01:00
Dag
c27a300e02 test: add failing mastodon test (#3255)
* fix: refactor cache factory

* test: add failing test

* add null cache
2023-02-15 21:22:37 +01:00
Joseph
787b4d7cad [DockerHubBridge] Add tag filter option (#3258)
* [DockerHubBridge] Add tag filter option

* [DockerHubBridge] Add example value

* [DockerHubBridge] lint

* [DockerHubBridge] Fix

* Update DockerHubBridge.php

* [DockerHubBridge] Make repo required

* [DockerHubBridge] Add filter example value for user images
2023-02-15 20:15:38 +01:00
Dag
17fcc72b09 chore: upgrade to php 8 in the Dockerfile, fix #3191 (#3256) 2023-02-13 22:21:07 +01:00
Joseph
67f72bfa5d [TelegramBridge] Support new username URL format (#3257)
* [TelegramBridge] Support new username URL format

* [TelegramBridge] Fix text error
2023-02-13 22:19:41 +01:00
sysadminstory
06a4bc4b45 [DealabsBridge-HotUKDealsBridge-MydealsBridge] (#3253)
Fix CSS class for the deal temperature
2023-02-10 06:16:01 +01:00
SergioFLS
ed36c8cbcd [ItchioBridge] add error for password pages (#3252) 2023-02-08 19:50:21 +01:00
sravan
4dc6158d5c fix badge (#3251) 2023-02-08 19:48:54 +01:00
Mynacol
38e832daae [GolemBridge] Remove link from author (#3248)
Fixes #3224.
2023-02-08 19:44:01 +01:00
Eugene Molotov
91f91ba621 [VkBridge] Follow site changes (#3244) 2023-02-07 15:10:43 +01:00
Dag
7f1b32f390 feat: add a proper feed item uid when the bridge errors out (#3237)
* refactor: move function to class

* fix: use the computed bridge name as cache key

* refactor: extract method

* fix: set a feed item uid on errors

* docs

* fix: remove year from uid
2023-02-02 22:53:01 +01:00
Dag
c97e1923ae docs (#3238) 2023-02-02 01:30:29 +01:00
Dag
e01e031e3a fix: # and / in filter bridge (#3236) 2023-02-01 20:11:20 +01:00
Bocki
1fe55314be [Core] Update GH Actions (#3235)
* [Core] Update GH Actions

* fix dockerbuild

* add empty lines at the end of files
2023-01-31 22:11:59 +01:00
Binnette
af6aedd536 php 7.4 (or higher) (#3232) 2023-01-30 22:42:39 +01:00
Sébastien MB
51952c1db2 [DevToBridge] Allow subsribe to username (#3218) 2023-01-30 11:59:46 +01:00
Justin Goette
69290c8e55 feat: [FilterBridge] Add URI/URL filter option to FilterBridge (#3212) 2023-01-29 20:13:29 +01:00
sysadminstory
57e79e7f1b [RadioMelodieBridge] Fix article content (#3230)
- remove the social network share section ath the beginning and and the
  end of the article
- removte the HTML formated article tilthe from the content
- limite the author picture size to 60 px
2023-01-29 19:45:55 +01:00
somini
b21806a0b3 [TheRedHandFiles]: New Bridge (#3229)
This blog had a feed, is updated irregularly, and someone disabled the
feed on the Wordpress config on purpose for some reason. :(
2023-01-28 07:39:59 +01:00
Mynacol
e44e458617 [HeiseBridge] add all RSS feeds from heise.de (#3223)
Add all feeds from https://www.heise.de/news-extern/news.html with
(mostly) their original name.

The existing ones moved from e.g. https://www.heise.de/newsticker/heise-atom.xml
to https://www.heise.de/rss/heise-atom.xml.

Some feeds were commented out, as they do not provide full-text feeds
for multiple reasons.
2023-01-27 02:54:08 +01:00
Korytov Pavel
e76ddf9192 [TldrTechBridge] Add bridge (#3222)
* [TldrTechBridge] Add bridge

* [TldrTechBridge] Fix lint

* [TldrTechBridge] Fix lint
2023-01-23 19:22:13 +01:00
sysadminstory
e1e340f650 [PicnobBridge] New Bridge (#3221)
Add new bridge
- some image can not be displayed in the feed because of the CORP policy
2023-01-22 16:00:18 +01:00
Dag
c06e471ae9 feat: add new bridge for picarto.tv (#3220) 2023-01-20 22:48:06 +01:00
Dag
04090bd84a docs (#3217) 2023-01-17 21:21:32 +01:00
Tone
b9af53beff New Bridge for Caschys Blog (#3215) 2023-01-17 17:06:49 +01:00
Tone
077fc4bc3c Update HeiseBridge.php (#3214)
loading smaller images instead the original ones with a size of around 20Mb
2023-01-17 17:01:09 +01:00
sysadminstory
eb94107c15 [RadioMelodie] Fix bridge (#3204)
This fix follows the change in the CSS of the website.
2023-01-03 19:26:56 +01:00
Eugene Molotov
eab82f0a26 [VkBridge] Add video thumbnails (#3206) 2023-01-03 19:26:05 +01:00
Justin Goette
b58f2ed338 docs: Fix formatting (#3205) 2023-01-03 19:25:36 +01:00
csisoap
5d8ed2df51 [ReutersBridge] Only include main headline from Top News feed (#3199) 2022-12-31 07:05:25 +01:00
Justin Goette
910ccd3ad8 docs: Fix typo (#3196) 2022-12-28 00:34:49 +01:00
csisoap
2a69c585b8 [YoutubeBridge] fix blank page when duration limit used (#3192) 2022-12-18 08:23:18 +01:00
Dag
936ae8cca3 various fixes (#3190)
* fix: Call to a member function parent() on null

* fix: notice

fixes Trying to get property plaintext of non-object at bridges/WikiLeaksBridge.php line 96

* fix: CommonDreamsBridge
2022-12-13 21:04:57 +01:00
Dag
a13c4624fb feat: custom http ua in AO3, fix #3188 (#3189)
* refactor

* feat: custom http ua in AO3, #3188
2022-12-13 09:53:42 +01:00
Jan Justi
933be15a77 [docker-entrypoint.sh] Add custom formats (#3179) 2022-12-10 21:25:25 +01:00
Štěpán Škorpil
b4f9d27424 Added StreamCzBridge (#3187)
* Added StreamCzBridge

* Fixed codestyle on StreamCzBridge
2022-12-10 15:02:11 +01:00
Ololbu
7d5698a75f [TapasBridge] New bridge (#3184)
* [TapasBridge] New bridge

* [TapasBridge] Delete comment

* [TapasBridge] Delete context

* [TapasBridge] Delete context again

* [TapasBridge] Convert double quotes do single quotes

* [TapasBridge] Fix some lint errors

* [TapasBridge] Fix indentation style

* [TapasBridge] Fix some lint errors
2022-12-09 11:05:08 +01:00
Dag
ec1b9a110d fix: radiofrance, #3077 (#3186) 2022-12-08 21:36:28 +01:00
Dag
69518e95c1 feat: add tags to codeberg bridge, fix #3177 (#3185)
* refactor

* feat: add tags to codeberg bridge, fix #3177
2022-12-08 21:21:17 +01:00
Julien Papasian
59d3011c86 Add Nautiljon bridge (#3181) 2022-12-08 20:48:10 +01:00
Eugene Molotov
fdf380bccd [VkBridge] Remove junky 'Show more' button (#3176) 2022-12-03 04:58:25 +01:00
Steve Joerger
0423e1f268 Issue 3174 - Add docker-compose instructions to main README.md (#3175)
* Added instructions to use docker-compose copied from wiki with extra step of docker-compose command.

* Added header to differentiate docker-compose section. Added missing step to browse to url.
2022-12-03 04:56:37 +01:00
Dag
55b294665c fix: remove duplicate apt install in dockerfile (#3172) 2022-12-02 03:43:10 +01:00
Niehztog
7cf64daac9 Adds new Bridge for kitso.io episode feed (#3169)
* Adds new Bridge for kitso.io episode feed

* linter fixes, changed item titles

* more linter fixes

* fix bridge uri

* linter fixes

* added sorting of items by timestamp, added parameters to spicify show

* trying to fix linter again

* linter doesnt like tenary operators

* fix whitespace
2022-11-30 00:59:24 +01:00
Juan Jose Pablos
ca667d731c Update VproTegenlichtBridge.php (#3168)
Fix lint error
2022-11-29 15:19:16 +01:00
TinyWiFi
d38cd05bea Added vern.cc Public Instance (#3164)
done on behalf of ~vern admins (https://vern.cc/admins)
2022-11-27 17:10:57 +01:00
vincentvd1
9863204fa3 Feat: Add bridge for Vpro tegenlicht (#3162)
* [VproTegenlichtBridge.php] Created bridge

* [VproTegenlichtBridge.php] Added fetch exception
2022-11-25 20:39:56 +01:00
Dag
592ea8faa4 fix: TheHackerNewsBridge (#3159) 2022-11-22 19:09:49 +01:00
ORelio
f53e0e4bee [DarkReading] Convert pictures to plain images (#3158)
Convert <picture> to <img> element
2022-11-22 18:42:31 +01:00
ORelio
d592e2cb15 [Core] Add html/convertLazyLoading (+ document stripRecursiveHTMLSection) (#3157)
* [core] Add html/convertLazyLoading($dom)

Looks for lazy-loading attributes such as 'data-src' and converts
them back to regular ones such as 'src', easier for RSS readers.
It also converts <picture> elements to plain <img> elements.

* [core] Document html/stripRecursiveHTMLSection()

Add documentation for that function (no code changes).

* [WordPressBridge] Use convertLazyLoading()

* [WordPressBridge] Unwrap image figures

<img> inside <figure> may not display on RSS readers.
This converts them back to <img>, without losing caption if present.

* [ZDNet] Convert lazy loading images

* [code] html/stripRecursiveHTMLSection: Fix typo
2022-11-20 12:41:59 +01:00
Eugene Molotov
2f7f13d9fe [core] Implement SetBridgeCacheAction to store bridge's cache remotely (#3156) 2022-11-20 12:41:38 +01:00
Eugene Molotov
8990aeb9f4 [core] Merge GET and POST args into request array (#3155) 2022-11-20 12:41:20 +01:00
Dag
745a7ba122 fix: TheHackerNewsBridge (#3154) 2022-11-19 00:25:31 +01:00
Dag
88766e6fde fix: produce smaller log records for http exceptions (#3153) 2022-11-18 21:36:06 +01:00
Dag
4ac2feb392 docs: improve docs (#3152) 2022-11-17 18:51:37 +01:00
Dag
ebb82849e9 fix: notice in telegram (#3151)
* refactor: telegram

* fix: notice in telegram
2022-11-17 18:06:56 +01:00
Dag
3ee2c7f918 feat: new bridge RumbleBridge (#3150) 2022-11-17 18:06:35 +01:00
Dag
130212fba5 feat: new list of default bridges (#3148) 2022-11-17 18:06:19 +01:00
Dag
2e6facd863 docs: more screenshots (#3149) 2022-11-17 18:05:44 +01:00
Dawid Wróbel
b017f75767 [OLXBridge] new bridge (#2944)
* [OLXBridge] new bridge

* [OLXBridge] option to limit to shipping offers only

* [OLXBridge] set the feed title according to search query

* [OLXBridge] Fix PHP notices

* [OLXBridge] Remove trailing slash from the URL

* [OLXBridge] filter out the imposed additional search categories

* [OLXBridge] limit search to 'new' OLX platform variants

* [OLXBridge] Parse date, add ID, description

Deep-crawl all results. Penalty is low, as we were doing this for almost all of the results, anyway, yet it allows to obtain a unique ID, an uncomplicated Date string and a description.

Requires ext-intl for parsing the date according to locale.

* [OLXBridge] Parse date, add ID, description

Deep-crawl all results. Penalty is low, as we were doing this for almost all of the results, anyway, yet it allows to obtain a unique ID, an uncomplicated Date string and a description.

Requires ext-intl for parsing the date according to locale.

* [OLXBridge] Images are optional, handle appropriately

* [OLXBridge] handle the ID coming from sibling auto-moto portal

* [OLXBridge] handle the photos coming from sibling auto-moto portal

* [OLXBridge] use meta property to find img URL

* [OLXBridge] handle the date coming from sibling auto-moto portal

* [OLXBridge] use simplified syntax to retrieve content attribute value

* [OLXBridge] handle the description coming from sibling auto-moto portal

* [OLXBridge] fix phpcs complaints

* [OLXBridge] add categories

* [OLXBridge] handle the categories coming from sibling auto-moto portal

* [OLXBridge] hint image MIME type

OLX images have no obvious extension

* [OLXBridge] Fix content formatting

* [OLXBridge] URL is pattern-checked, so no need to check again

* [OLXBridge] return actual search query as URI
2022-11-17 17:57:05 +01:00
Dawid Wróbel
0726cce426 [AllegroBridge]: new bridge (#2942)
* [AllegroBridge]: new bridge

* [AllegroBridge] set feed name

* [AllegroBridge] fix notices

* [AllegroBridge] add images as attachments

* [AllegroBridge] cleanup code

* [AllegroBridge] add UID

* [AllegroBridge] add categories

* [AllegroBridge] fix pretty formatting

* [AllegroBridge] fix notice when offerExtraInfo is not found

* [AllegroBridge] add support for session cookie to prevent rate limiting

* [AllegroBridge] return inputted URL as URI

* [AllegroBridge] fix phpcs warning
2022-11-17 05:13:33 +01:00
Austin Huang
e788e14baa [MastodonBridge] Add support for various platforms (#3133)
* [MastodonBridge] Add support for various platforms

* [MastodonBridge] satisfy the lint
2022-11-16 18:05:01 +01:00
Dag
a5779d30b5 feat: add max file size to http responses (#3140) 2022-11-16 17:56:26 +01:00
Dawid Wróbel
c2c88e9876 Dockerfile: Use libcurl-impersonate in place of libcurl (#2941)
Fixes #2547
2022-11-15 17:32:44 +01:00
Dag
dbab225fd2 fix: Call to a member function find() on bool (#3146)
* fix: Call to a member function find() on bool

Happens when defaultLinkTo() is passed the empty string.

* fix: prevent exception in defaultLinkTo() when passed the empty string

* refactor
2022-11-15 03:01:27 +01:00
Dag
001427243c feat: system alert message (#3139) 2022-11-15 00:32:04 +01:00
Dag
95c199c2eb fix: various php notices (#3145)
* fix: notice

* fix: Trying to get property content of non-object at bridges/PcGamerBridge.php line 36

* fix: better exception message

* fix: strpos(): Non-string needles will be interpreted as strings in the future. Use an explicit chr() call to preserve the current behavior
2022-11-15 00:30:51 +01:00
Joseph
734a5868b8 [FirefoxAddonsBridge] Fix removal of link redirects (#3144)
* [FirefoxAddonsBridge] Fix removable of link redirects

* Update FirefoxAddonsBridge.php
2022-11-14 22:56:12 +01:00
Paroleen
80f9871c9e [AwwwardsBridge] Fix sites parsing (#3141)
* [AwwwardsBridge] Fix sites parsing

* [AwwwardsBridge] Fix phpcs issue
2022-11-11 18:33:28 +01:00
Dag
b64f8f2a09 fix: various fixes (#3136) 2022-11-08 21:17:32 +01:00
quickwick
00ff0890bb Add item dimensions (width x height) and source URL (if it exists in JSON) (#3135) 2022-11-08 18:43:51 +01:00
Dag
86c3a969b8 feat: add admin telegram contact config (#3134) 2022-11-07 18:36:52 +01:00
Dag
2ef98b299f refactor: extract frontpage to template (#3130)
Also introduce usage of Response object
2022-11-07 18:22:54 +01:00
MarKoeh
fe59cbabc9 [ARDAudiothekBridge] added bridge ARDAudiothek.de (#3132)
* [ARDAudiothekBridge] added bridge ARDAudiothek.de

ARD, the union of Germany's regional public-service broadcasters, operates a video and an audio streaming service. The video streaming service is ARDMediathek, for which a bridge already exists. The audio streaming service is ARDAudiothek. This commit adds initial support for ARDAudiothek. It currently supports turning shows to feeds.

* [ARDAudiothekBridge]  fixed code style

Sorry. Forgot spaces surrounding the concatenation symbol
2022-11-04 19:03:12 +01:00
Alexander
aa14d4aafb [SteamAppNewsBridge] new bridge (#3126)
* [SteamAppNewsBridge] new bridge

* [SteamAppNewsBridge] test fixes
2022-11-03 21:43:33 +01:00
Dag
75772f58e2 fix: review news was not properly extracted (#3131)
Fix #3129
2022-11-03 21:42:26 +01:00
John S Long
400e137673 [DiscogsBridge] Add optional image, if personal access token is configured (#3083) 2022-11-03 20:33:43 +01:00
ORelio
4520ab6835 [WordPressBridge] Improve content extraction (#3125)
* [WordPressBridge] Improve content extraction

 - Pick up currently unmaintained bridge
 - Allow Custom item limit and lower default limit from 20 to 10
 - Allow Custom content selector for blogs with non-standard templates (#2173)
 - Remove content selector made for one specific blog (#2173 - can be a custom selector now)
 - Add '.article-content' class in the set of default selectors
 - Improve lazy-loading conversion

* [WordPressBridge] Fix phpcs issues
2022-10-31 20:59:19 +01:00
Dag
7250940a05 fix: prtester css replacement (#3123) 2022-10-29 12:25:45 +02:00
Eugene Molotov
8779c09e89 [PikabuBridge] Remove html elements from feed item author (#3122) 2022-10-29 12:14:12 +02:00
Dag
23f8c81646 refactor/fix: css organization and error rendering (#3117)
* fix: php notice

* refactor/feat: merge HtmlFormat.css into style.css

Also improve ux of error rendering.

* fix: center-align footer text
2022-10-29 10:46:37 +02:00
Dag
1b45a53402 fix: make filecache be case-sensitive on key (#3113) 2022-10-29 10:27:26 +02:00
Dag
e027bd9274 fix: improve FeedExpander (#3103)
* fix: improve FeedExpander

Include the first libxml error in exception.

Give better error message if trying to parse the empty string.

Log all libxml errors if debug mode is enabled.

* error handling and logging tweak
2022-10-29 10:27:02 +02:00
Dag
f9cd397900 Update README.md (#3121) 2022-10-29 10:21:51 +02:00
Dag
1f576312ea feat: in debug mode, include part of http response in exception message (#3090) 2022-10-29 08:42:50 +02:00
Justin Goette
85b87b9597 docs: Document returnFull parameter of getContents (#3120) 2022-10-29 03:30:49 +02:00
joaomqc
bef6fc5cbd [SpotifyBridge] Add playlists feed (#3116)
* [SpotifyBridge] Add playlists feed

* fix formatting

* remove whitespace

* merge artist and playlist feeds

* fix lint errors
2022-10-27 20:02:01 +02:00
Dag
314d4c7a3f fix: quickfix for heise, #3118 (#3119) 2022-10-27 19:59:45 +02:00
Dag
52af2ae34c fix: php errors (notices) (#3115) 2022-10-26 00:47:45 +02:00
Juan Jose Pablos
8795cb252f A bit of info on cookies. I need to expand more. (#3111)
* add a bit of info about cookies

* Use markdown to look nice

* Update FacebookBridge.md

Fix url
2022-10-25 14:32:47 +02:00
Dag
cf7896aeef fix: forever loop in golem (#3114) 2022-10-25 14:30:01 +02:00
ORelio
05f2fb5ec7 [FeedExpander] Decode HTML entities in title (#3110)
Feed item title may contain HTML entities that we need to decode,
else they are encoded twice when generating the expanded feed.
2022-10-20 18:26:43 +02:00
Eugene Molotov
d483bf2b81 [core] Implement bearer token authentication (#3043) 2022-10-19 18:39:35 +02:00
Korytov Pavel
5b53e76477 [ScientificAmericanBridge] Add bridge (#3109)
* [ScientificAmericanBridge] Add bridge

* [ScientificAmericanBridge] Fix lint errors and timeout
2022-10-19 18:34:11 +02:00
Juan Jose Pablos
6ae40496c8 docs: add facebook bridge page 2022-10-18 19:35:53 +02:00
Dag
58e321c915 fix: flickr (#3104) 2022-10-17 00:38:57 +02:00
Dag
37f1ab726b fix: various bug fixes (#3102)
* fix: Undefined offset: 4

* fix: Trying to access array offset on value of type bool

* fix: Undefined variable: photo at bridges/TelegramBridge.php line 287

* fix: Trying to get property innertext of non-object at bridges/ZDNetBridge.php line 186

* fix: Undefined index: Category at bridges/UnraidCommunityApplicationsBridge.php line 42

* fix: Undefined index: fullUrl at bridges/EuronewsBridge.php line 61
2022-10-16 20:26:33 +02:00
Dag
ffbc107687 Improve logging and error handling (#3059)
* refactor: logging and errror handling
2022-10-16 17:55:43 +02:00
Dag
e21394d2d3 refactor: html format (#3062)
* refactor: html format

Fix a few small bugs too

* fix

* fix

* trigger build

* striptags instead of encode title
2022-10-16 12:03:57 +02:00
John S Long
78fa03238c [MastodonBridge] Add support for GoToSocial (#3098)
* [MastodonBridge] Add support for GoToSocial

GoToSocial expects URLs in HTTP Signatures to include the query string; Mastodon
does not. To provide support for both types of ActivityPub services, define a
new parameter for signature types, defaulting to Mastodon's format.

This change also introduces auto-resolution of linked objects, which GoToSocial
uses instead of including content directly in a user's outbox.

* [MastodonBridge] Fix lint failure
2022-10-12 21:43:09 +02:00
Harvey Christian Pacleb
4d8e40e746 [HonkaiImpactSeaBridge] Add bridge (#3084) 2022-10-12 21:29:38 +02:00
floviolleau
3ea7d46837 [PanneauPocketBridge] enhancements (#2940)
* [PanneauPocketBridge] small fixes

* [PanneauPocketBridge] rename variable

* [PanneauPocketBridge] remove function call

Co-authored-by: Florent VIOLLEAU <florent.violleau@samsic.fr>
2022-10-12 21:10:58 +02:00
An | Anton Röhm
7895fa895f fix "cht on matrix" button (again) (#3101) 2022-10-12 20:48:34 +02:00
John S Long
5e664d9b2b [Dockerfile] Enable opcache for improved performance (#3097)
Opcache is a PHP extension that caches the bytecode PHP converts each script
into to reduce the work that needs to happen each request.
2022-10-09 21:14:24 +02:00
Korytov Pavel
56a8c521c2 [EconomistBridge] Fix bridge (#3095) 2022-10-08 20:05:17 +02:00
Christian Schabesberger
79f6ec5733 hide Nordbayern+ articles (#3094) 2022-10-08 15:34:26 +02:00
Dag
0a2e31e1f7 docs: sort public instances (#3093) 2022-10-06 22:12:55 +02:00
Dag
e65fd7c822 fix: remove debug line (#3092) 2022-10-06 21:05:50 +02:00
Wouter Koch
c4c2acab98 Add NOS Nieuws & Sport Bridge (#3069)
* Add NOS Nieuws & Sport Bridge

* Change classname to reflect filename (NOSBridge)
2022-10-05 19:41:21 +02:00
joshinat0r
8b7b32d516 [RedditBrige] Fix old feed URLs (#3087)
* flair filter

* syntax

* fix multi & user feeds

* '

* dont replace ,

* fix old reddit feeds
2022-10-05 19:35:02 +02:00
Dag
5d18852108 fix: more verbose error in fb (#3089) 2022-10-05 19:30:42 +02:00
Dag
44e5bf5338 docs: point to offical feed in qnap (#3088) 2022-10-05 19:21:02 +02:00
somini
8b91921a70 [AsahiShimbunAJWBridge]: Fix title extraction (#3085) 2022-10-04 21:23:49 +02:00
Nicolas Delsaux
1232de5744 Add new bridge RadioFranceBridge, fix #3077 (#3082)
As far as I understand, this supports all radio france sub-pages listing podcast and/or shows (tested with my two favorites shows, so not a very professionnal test).
We use here the data model provided by Radio France, which includes all data in an easily usable format.
2022-10-03 06:55:24 +02:00
John S Long
6b83bf25fd [RoosterTeethBridge] Add episode description, optional episode image (#3080) 2022-10-02 19:38:33 +02:00
joshinat0r
30d964b356 [RedditBridge] Search for specific flairs (#3067)
* flair filter

* syntax

* fix multi & user feeds

* '

* dont replace ,
2022-10-02 07:34:20 +02:00
John S Long
8dcc21a871 [RoosterTeethBridge] Add new channels (#3076) 2022-10-02 06:48:07 +02:00
John S Long
7252a89914 [LWNprevBridge] Fix article content parsing (#3078) 2022-10-02 06:46:04 +02:00
An | Anton Röhm
3198a48589 Fix broken "chat on matrix" button (#3075) 2022-10-01 08:43:34 +02:00
Corentin Garcia
7c96334e3b [GithubSearchBridge] Add programming language (#3074) 2022-10-01 08:42:50 +02:00
Corentin Garcia
c4d95558af [GenshinImpactBridge] fix missing articles (#3073) 2022-10-01 08:42:02 +02:00
Corentin Garcia
8ac5045963 [TheGuardianBridge] Fix missing article content (fix #3032) (#3072) 2022-10-01 08:41:19 +02:00
Corentin Garcia
4a21855e5c [EliteDangerousGalnetBridge] Fix missing news (#3071) 2022-10-01 08:40:57 +02:00
Tobias Alexander Franke
1ffd9ee61a [GoogleScholarBridge] Follow authors of scientific publications. (#3066)
* [GoogleScholarBridge] Follow authors of scientific publications.

* [GoogleScholarBridge] Fix linting.

* [GoogleScholarBridge] Fix more linting issues.

* [GoogleScholarBridge] Use author field in extracted page and drop publisher
2022-09-25 22:22:49 +02:00
Tobias Alexander Franke
faf63269a1 [BinanceBridge] Adapt code to new JSON structure and fetch full article (#3065)
* [BinanceBridge] Add new bridge

* [BinanceBridge] TravisCI fixes

* [BinanceBridge] PR fixes

* [BinanceBridge] Fix for Binance blog: Pull JSON data instead of HTML

* [BinanceBridge] Fix double quotes

* [BinanceBridge] Remove announcements category (because of Cloudflare)

* [BinanceBridge] Simplify code to bare minimum

* [BinanceBridge] Adapt code to new JSON structure and fetch full article

* [BinanceBridge] Fix linting issues
2022-09-25 19:19:35 +02:00
Dag
8cc5e44be6 fix: use parlers new api (#3061) 2022-09-24 00:02:19 +02:00
Dag
aacba5b1a8 fix: too strict url validation in feed item (#3058)
Urls such as https://example.com/réponse were rejected

Fix https://github.com/RSS-Bridge/rss-bridge/issues/3018#issuecomment-1254159203
2022-09-21 23:07:56 +02:00
Mynacol
9d871e8a45 [ZeitBridge] Add bridge for zeit.de (#3056)
* [ZeitBridge] Add bridge for zeit.de

New bridge expanding the feeds of zeit.de to full-text ones.
Circumvents cookie banners and Z+ premium article paywalls.

* [ZeitBridge] Formatting
2022-09-21 22:24:11 +02:00
Dag
aabbeef743 docs: add a better bridge example in readme (#3057) 2022-09-21 22:20:51 +02:00
Mynacol
8d8fe66aab [HeiseBridge] Parser rewrite (#3054)
* [HeiseBridge] Parser rewrite

This rewrite is more readable and consistent than the previous one.

Additionally, this removes unwanted elements, largely recommendations
for other articles.
Furthermore, it increases the image quality by using the original
picture link instead of the compressed ones.

* [HeiseBridge] Formatting
2022-09-21 21:31:43 +02:00
Eugene Molotov
3d9fead463 [ActionFactory] Allow camel-case action names (#3044)
Dash symbol is used to convert dash-seperated string to camel-cased string
2022-09-20 18:17:57 +02:00
Eugene Molotov
2db523a37a [VkBridge] Handle empty posts before fixing image links (#3052) 2022-09-20 18:07:57 +02:00
Eugene Molotov
6cd8b90d28 [VkBridge] Follow changes on HTTP redirection (#3051)
When visiting canonical link like https://vk.com/club1,
VK returns redirection response to non-canical link,
which raises "Unexpected redirect location" exception.

This patch removes path check in order to handle this situation
2022-09-20 18:07:31 +02:00
Tobias Zulauf
f660c16ca6 [TwitterV2Bridge] noexternallink option to remove external stuff from the content_html output (#3041)
* [TwitterV2Bridge] noexternallink option to remove external stuff from the content_html output

* Update bridges/TwitterV2Bridge.php

* Update bridges/TwitterV2Bridge.php
2022-09-20 18:07:18 +02:00
Eugene Molotov
a12152e8a5 [VkBridge] Code cleanup (#3047)
- Remove .page_album_link patch, since VK already patched its renderer
- Remove non working code, that tries to get clean video links
2022-09-19 19:22:41 +02:00
Dag
55cc74c816 feat: new bridge pokemonnews (#3042)
* feat: new bridge pokemonnews

fix #3040

* fix
2022-09-17 01:29:40 +02:00
Eugene Molotov
11220ef373 [VkBridge] Photo fixes (#3039)
This commit fixes following issues:

- Photos from user profile wall started appearing as blured
- On posts with photo collection small thumbnails are shown
2022-09-15 22:17:10 +02:00
quickwick
a1e229a7e1 feat: check EZTV mirrors for available site to query (#3036) 2022-09-15 17:47:57 +02:00
Sébastien SAUVAGE
925caff5dd Merge pull request #3037 from RSS-Bridge/sebsauvage-remove-public-host-sebsauvage.net
Removing public instance on sebsauvage.net
2022-09-15 11:35:15 +02:00
Sébastien SAUVAGE
0bf41a5726 Removing public instance on sebsauvage.net 2022-09-15 11:33:16 +02:00
Dag
6f7be67a8c fix: broken bridge abcnews (#3033)
Fix #3031
2022-09-13 19:00:51 +02:00
Dag
2d272117cc fix: dont error out for invalid env names (#3030) 2022-09-12 23:14:11 +02:00
Dag
5a9336df12 fix: wrong accept header in mastodonbridge (#3025)
Fixes bug introduced by afcc38786e because of
the default Accept header which caused xml to be returned.

Fix #3024
2022-09-10 07:38:09 +02:00
Dag
94ae098ef5 fix: various fixes (#3023)
* improve twitch error message

* fix worldcosplay notice

* fix: add new video image to telegram

* fix: reuters

* fix: formula1

* twitter
2022-09-09 20:18:50 +02:00
Dag
6ac347d5ac fix: add workaround for NYTBridge antibot (#3022) 2022-09-08 19:54:09 +02:00
Dag
27b3d7c34e feat: improve logging and error handling (#2994)
* feat: improve logging and error handling

* trim absolute path from file name

* fix: suppress php errors from xml parsing

* fix: respect the error reporting level in the custom error handler

* feat: dont log error which is produced by bots

* ignore error about invalid bridge name

* upgrade bridge exception from warning to error

* remove remnants of using phps builin error handler

* move responsibility of printing php error from logger to error handler

* feat: include url in log record context

* fix: always include url in log record contect

Also ignore more non-interesting exceptions.

* more verbose httpexception

* fix

* fix
2022-09-08 19:07:57 +02:00
Dag
5578a735d9 feat: allow more feeds in FeedMerge (#3021)
Fix #3011
2022-09-08 18:44:15 +02:00
Dag
e63e3d072c feat: new bridge QnapBridge (#3020) 2022-09-08 18:28:36 +02:00
Eugene Molotov
70ba6c5b53 [VkBridge] Manually handle redirects (#3017)
Some redirects are legit, some redirects lead to "Too many requests" page
2022-09-07 03:02:23 +02:00
Eugene Molotov
bbbd599bc8 fix: do not throw exception on 301-303 http responses (#3014) 2022-09-06 15:40:20 +02:00
Dag
0dab51e26f fix: php errors (#3013)
* fix: php error in tiktok bridge

* fix: notice in craigslist

* fix: php notice in wordpress bridge

* feat: improve ux in telegram bridge
2022-09-06 00:14:20 +02:00
Eugene Molotov
8033a5f461 [VkBridge] Fix photo URI retrieving (#3010)
VK stopped filling JSON structure containing information for generating URI to full size photo
2022-09-05 23:41:41 +02:00
Dag
8ea9472300 feat: improve exception message when xml parsing fails (#3009) 2022-09-05 14:26:11 +02:00
Dag
a2c7865226 fix: exclude mastodon boosts when told to do so (#3007) 2022-09-05 05:58:18 +02:00
quickwick
36f64a3258 feat: add preview for external urls in twitterv2 (#3006)
Fix #2430
2022-09-05 05:31:36 +02:00
Dag
51f6ec8dc1 Update README.md (#3005) 2022-09-05 02:32:06 +02:00
quickwick
8ffc002e53 fix: truncated retweet text in twitter bridge(#3004)
Fix #2988
2022-09-05 02:04:09 +02:00
Dag
cca11174e1 fix: php error in ViceBridge (#3003)
Fixes Undefined variable: article at bridges/ViceBridge.php line 37
2022-09-04 22:58:58 +02:00
Dag
9d16e81e17 fix: php error PHP Notice: Undefined index: author (#3002) 2022-09-04 19:07:04 +02:00
Dag
5e09a14acc fix: include git tag in version (#3000)
Also include os and php version in github issue template.
2022-09-04 07:21:57 +02:00
Dag
693d6edfbf fix: php7.3 parser error in Configuration.php (#2999)
Dont fail a basic lint so that the proper error message can be shown when on unsupported versions.
2022-09-04 04:50:01 +02:00
Dag
57d5aa45f7 fix: php notice in eztvbridge (#2998)
* fix: php notice in eztvbridge

Fixes Undefined property: stdClass::$torrents

* lint
2022-09-04 04:35:21 +02:00
Dag
b8f73618c1 fix: include playlist when processing soundcloud items (#2997) 2022-09-04 03:50:40 +02:00
Dag
f40ed566be fix: absolute urls for images in pixivbridge (#2993) 2022-09-02 20:35:17 +02:00
Dag
3c2353c0ec fix: bug in previous refactor (#2992)
fix #2991
2022-08-31 18:16:19 +02:00
Dag
97808abca1 refactor: rename rssbridge.php to bootstrap.php (#2987)
Fix #2986
2022-08-27 23:01:06 +02:00
Korytov Pavel
1ca4dd69f7 [InstituteForTheStudyOfWarBridge] Do not put HTML tags into title (#2985) 2022-08-25 19:48:16 +02:00
Korytov Pavel
c079dbb521 [InstituteForTheStudyOfWarBridge] Add bridge (#2984) 2022-08-25 19:22:19 +02:00
Eugene Molotov
43ad54dba0 [PikabuBridge] Skip sponsored posts (#2983)
* [PikabuBridge] Skip sponsored posts

Sponsored posts appear very rarely in html code.
But when they appear, they always have different url that results junk feed.

One of the example is [1]. After visiting it, you will be redirected to [2] that is marked as
"Партнёрский материал" in Russian, or "Sponsored post" in English.

[1] https://pikabu.ru/story/a_mla_posa_m_memu_seyla_otorathche_idomikhlenonoikhmyav_sseyla_otoratazoed__9388770?from=cpm
[2] https://pikabu.ru/story/kakim_dolzhen_byit_vash_noutbuk_9388770

* lint

Co-authored-by: Dag <me@dvikan.no>
2022-08-25 18:09:06 +02:00
Eugene Molotov
fd0d5350be [RutubeBridge] Include timestamp and author in feed (#2982) 2022-08-24 14:49:54 +02:00
Dag
5165ea265d fix: case-insensitive config from env, fix #2935 (#2973)
* refactor

* fix: case-sensitive config from env, fix #2935

* lowercase all config section and keys

* test: add test for case-insensitivity
2022-08-23 21:19:53 +02:00
Bocki
edbafc6144 [Teefury] Fix occasional problem (#2974) 2022-08-20 21:11:27 +02:00
Dag
0de1694371 refactor: extract index.php to RssBridge.php (#2961)
* refactor: extract entry point into class

* refactor: js

* refactor: extract frontpage action
2022-08-18 22:52:01 +02:00
Christian Schabesberger
372eccd7b2 [Nordbayern] Update regions (#2966) 2022-08-13 19:41:05 +02:00
Bocki
e99cbf21b2 [DailyShirts] Add daily shirt sites (#2962) 2022-08-12 14:51:38 +02:00
Miika Launiainen
08c1f55f4a Created Hanime bridge (#2958)
* Created Hanime bridge

* Moved cover image from enclosures to content as dvikan suggested
2022-08-10 23:46:17 +02:00
Dag
6253538342 docs: revamp README (#2954) 2022-08-06 23:22:50 +02:00
Dag
502799a74c feat: use bridge description and short name in search (#2952)
* refactor: search.js

* feat: use bridge description and short name in search

* fix bug in previous merge commit

Also reformat string from tabs to spaces
2022-08-06 23:12:30 +02:00
Dag
eef45d4e8d fix: TypeError (0): setlocale(): Argument #1 ($category) must be of type int, string given (#2951)
This was upgraded from a warning to an error in php 8.
2022-08-06 22:46:49 +02:00
Dag
2bbce8ebef refactor: general code base refactor (#2950)
* refactor

* fix: bug in previous refactor

* chore: exclude phpcompat sniff due to bug in phpcompat

* fix: do not leak absolute paths

* refactor/fix: batch extensions checking, fix DOS issue
2022-08-06 22:46:28 +02:00
Dag
b042412416 fix: force HTTP 1.1 in curl requests (#2949)
Since curl 7.62.0 the default option is: CURL_HTTP_VERSION_2TLS

Before that the default used to be: CURL_HTTP_VERSION_1_1

Fix #2947
2022-08-05 11:46:32 +02:00
Eugene Molotov
205f0a7239 [RutubeBridge] Fix regex for retreiving reduxState (#2955)
Before this commit regex captured window.reduxState value until first semicolon.
This is incorrect since it produces invalid json, if semicolon is
also somethere in the middle of stringified json.

After this commit regex will capture window.reduxState value until last semicolon.
2022-08-05 11:45:50 +02:00
Dag
3984427f44 feat: include os and php version in github issue body (#2948) 2022-08-03 17:05:13 +02:00
Dag
ecb486794b refactor: use static values for cache scope
This fixes a future problem when code is placed under a namespace because `get_class($bridge)` will then return e.g. `RssBridge\Bridge\TwitterBridge` instead of the the current value `TwitterBridge`.

Also a bit refactoring of `Configuration.php`.
2022-08-02 15:03:54 +02:00
Loïc Fürhoff
a0a0d5235b Remove MAINTAINER (#2946) 2022-07-31 21:42:40 +02:00
Dag
afcc38786e fix: use default headers in getContents() (#2927) 2022-07-31 04:21:56 +02:00
Dag
cd0ca7f645 fix: change default curl user agent (#2926) 2022-07-31 03:58:07 +02:00
Dag
0a060b2ad6 [Gab] feat: add new bridge GabBridge (#2920) 2022-07-31 03:52:27 +02:00
Jan Tojnar
5b5f3b4254 Do not use constants for configuration (#2938)
* docs: Do not use constant names when referring to config options

The options are customizable using a config file and no longer hardcoded in index.php since 8ac8e08abf

* Do not use constants for configuration

Since <8ac8e08abf>, they are just set to the configuration object values.
2022-07-24 19:26:12 +02:00
Dawid Wróbel
499d5c2b77 [Amazon & AmazonPriceTracker] Add Poland (#2930) 2022-07-21 20:41:15 +02:00
llamasblade
2c63d5707d [HytaleBridge] Improve bridge performance (#2928) 2022-07-21 20:33:00 +02:00
llamasblade
9b0f8095c2 [YandexZenBridge] Fix feed title if username not specified (#2922) 2022-07-13 12:08:11 +02:00
llamasblade
1294d3b953 [YandexZenBridge] Add bridge (#2921) 2022-07-13 01:08:05 +02:00
llamasblade
64c8d4ad37 [HytaleBridge] Improve bridge contents (#2912)
Co-authored-by: BuildTools <unconfigured@null.spigotmc.org>
2022-07-10 21:06:41 +02:00
Dag
5e52ecc3f8 test: add new test for Configuration (#2915) 2022-07-10 20:05:27 +02:00
Dag
c33f84fcc2 fix: disallow non-strings in GET parameters (#2908) 2022-07-10 19:50:51 +02:00
Dag
003ab58514 [FurAffinity] fix: errror (#2887)
They changed parts of the dom.

Fix error:
Error: Call to a member function find() on null

Fixes #2868
2022-07-10 19:48:37 +02:00
Loïc Fürhoff
87f8571ccf [Mailman2] Add bridge (#2877) 2022-07-10 19:40:03 +02:00
floviolleau
f1319f5b2b [PanneauPocket] add new bridge (#2823) 2022-07-10 19:36:01 +02:00
Dag
40dc0a2e5f [Euronews] fix: use correct url (#2916)
The non-www domain has a tls config error.
2022-07-09 22:50:03 +02:00
Bocki
e89329b2c6 [core] Fix daux generation (#2914) 2022-07-09 20:33:07 +02:00
Jan Tojnar
e07a94d480 Normalize some method calls (#2911)
The methods were called as static even though they were not.
2022-07-09 08:13:07 +02:00
Dag
a966213cd7 refactor: inject the action params via its execute method (#2907) 2022-07-08 21:06:14 +02:00
Jan Tojnar
22c10941dc docs: Update directory structure description (#2906)
- The css directory was moved in 1a4c3f4418
2022-07-08 21:00:32 +02:00
Dag
abc4af43b3 feat: improve error handling (#2902) 2022-07-08 20:39:13 +02:00
Dag
c992bcc8bf [AssociatedPressNews] fix: prepend lead photo to items (#2905) 2022-07-08 18:42:45 +02:00
Jan Tojnar
f672902896 Add .git-blame-ignore-revs file (#2903)
This will ignore coding style change commits in GitHub’s blame UI.

Same thing can be achieved locally using either `git blame --ignore-revs-file .git-blame-ignore-revs`
or `git config blame.ignoreRevsFile .git-blame-ignore-revs`

https://git-scm.com/docs/git-blame#Documentation/git-blame.txt---ignore-revs-fileltfilegt
https://docs.github.com/en/repositories/working-with-files/using-files/viewing-a-file#ignore-commits-in-the-blame-view
2022-07-08 17:00:34 +02:00
Dag
abfc6b4633 feat: introduce template engine (#2899) 2022-07-08 14:17:25 +02:00
Jan Tojnar
951092eef3 Fix coding style missed by phpbcf (#2901)
$ composer require --dev friendsofphp/php-cs-fixer

$ echo >.php-cs-fixer.dist.php "<?php

$finder = PhpCsFixer\Finder::create()
    ->in(__DIR__);

$rules = [
    '@PSR12' => true,
    // '@PSR12:risky' => true,
    '@PHP74Migration' => true,
    // '@PHP74Migration:risky' => true,
    // buggy, duplicates existing comment sometimes
    'no_break_comment' => false,
    'array_syntax' => true,
    'lowercase_static_reference' => true,
    'visibility_required' => false,
    // Too much noise
    'binary_operator_spaces' => false,
    'heredoc_indentation' => false,
    'trailing_comma_in_multiline' => false,
];

$config = new PhpCsFixer\Config();

return $config
    ->setRules($rules)
    // ->setRiskyAllowed(true)
    ->setFinder($finder);

"

$ vendor/bin/php-cs-fixer --version
PHP CS Fixer 3.8.0 BerSzcz against war! by Fabien Potencier and Dariusz Ruminski.
PHP runtime: 8.1.7

$ vendor/bin/php-cs-fixer fix
$ rm .php-cs-fixer.cache
$ vendor/bin/php-cs-fixer fix
2022-07-08 13:00:52 +02:00
Jan Tojnar
dbf8c5b7ae refactor(BridgeFactory): make methods only accept valid class names (#2897)
This moves the responsibility for getting a valid class name
to the users of BridgeFactory, avoiding the repeated sanitation.
Improper use can also be checked statically.
2022-07-08 12:54:23 +02:00
sal0max
20bf2aa4fe [ExplosmBridge] merge ExplosmBridge and CyanideAndHappinessBridge (#2844) 2022-07-08 00:23:29 +02:00
llamasblade
f887ce8f63 [HytaleBridge] Add bridge (#2900) 2022-07-07 15:12:35 +02:00
Tokariew
ea45717a28 [Instagram] fix: add ds_user_id (#2881)
Fix #2876
2022-07-07 12:08:21 +02:00
Jan Tojnar
d107f8ed30 Improve Factory variable names (#2895) 2022-07-06 12:14:04 +02:00
Bocki
e3dad86bca [core] prtester fix for optgroups (#2896) 2022-07-06 11:26:53 +02:00
Joseph
6c52e9bbc6 [TelegramBridge] Support telegram.me in detect params regex (#2891) 2022-07-06 03:48:49 +02:00
Jan Tojnar
e254dfbb9c ci: Fix PHPCompatibility again (#2892)
The fix in 66568e3a39 prevented an error
when installing phpcompatibility/php-compatibility but there was still
a warning before that when installing dealerdirect/phpcodesniffer-composer-installer.
With Composer 2.3.9, this is now an error too, so we need to move
the config change before that: https://getcomposer.org/changelog/2.3.9
2022-07-06 03:39:58 +02:00
Jan Tojnar
b444fa71f5 docs: Update requirements
This was forgotten in 8365a7a34d.
2022-07-06 03:34:37 +02:00
Jan Tojnar
ab6aca3163 lib/Configuration: Remove redundant comment
It was just getting out of sync:

- Minimum PHP version was bumped in 8365a7a34d
- Cache directory permission check was removed in 8e2b65556f
- Whitelist permission check was removed in d4e867f240
2022-07-06 03:34:37 +02:00
Jan Tojnar
7ee942621d composer: Update lockfile
composer.json was modified in 8365a7a34d
but the changes were not propagated to the lockfile,
resulting in warnings on every installation.
2022-07-06 03:34:37 +02:00
Dag
192fc0ee9b [FeedMerge] feat: remove duplicates (#2888)
Fix #2855
2022-07-05 15:39:00 +02:00
Dag
321ec7c8c1 refactor: move cache logic into the factory (#2884) 2022-07-05 13:20:01 +02:00
Dag
5b9b579652 refactor: remove unused class (#2883) 2022-07-05 10:50:39 +02:00
Dag
e918bda735 chore: introduce CONTRIBUTORS.md (#2839) 2022-07-04 07:33:23 +02:00
sal0max
7d941c2898 [Flaschenpost] Add bridge (#2808) 2022-07-04 07:29:22 +02:00
Dag
4f75591060 Reformat codebase v4 (#2872)
Reformat code base to PSR12

Co-authored-by: rssbridge <noreply@github.com>
2022-07-01 15:10:30 +02:00
Jan Tojnar
66568e3a39 ci: Fix PHPCompatibility (#2873)
> For additional security you should declare the allow-plugins config with a list of packages names that are allowed to run code. See https://getcomposer.org/allow-plugins
> You have until July 2022 to add the setting. Composer will then switch the default behavior to disallow all plugins.

Oops, it is July now.
2022-07-01 15:02:04 +02:00
Dag
9f2f1e526d [Instructables] refactor: conform to PSR2 (#2870) 2022-07-01 02:19:47 +02:00
Patrick Collins
2c7a9d7c45 [MangaDex] improve date handling (#2864)
prioritize new chapters rather than just edited ones.
should avoid batch-renaming drowning out new chapters, and existing items being re-sorted in the feed when they're edited.
documentation here: https://api.mangadex.org/docs/dates/
2022-06-26 08:50:51 +02:00
Dag
5076d09de6 refactor: prepare for PSR2 (#2859) 2022-06-24 18:29:35 +02:00
Dag
d2313bddcc feat: print which bridge is being processed (#2860) 2022-06-24 14:40:17 +02:00
Jan Tojnar
b1ae7603bf ci: Install PHPUnit as a Composer dependency (#2857)
Now that we dropped support for deprecated PHP versions,
we can use the same PHPUnit version on all supported PHP version.
Let’s install it as a Composer dependency to have the same
PHPUnit version on the CI as on developers’ computers.
2022-06-24 12:00:58 +02:00
KamaleiZestri
556f0b4237 [NewgroundsBridge] Add Bridge (#2849) 2022-06-24 11:59:24 +02:00
Yaman Qalieh
1e7ad6ed51 [PixivBridge] Remove backwards compatability hack (#2846)
There is no need for this since the context is guessed if it's not
provided.
2022-06-24 11:31:24 +02:00
Dag
92782e6c34 [GoogleSearch] feat: add verbatim option (#2858)
The verbatim option is the same as wrapping the query in quotes e.g. "rss-bridge".
2022-06-24 11:18:27 +02:00
Yaman Qalieh
1cfcacbbeb [BugzillaBridge] Add new bridge (#2825) 2022-06-24 11:04:49 +02:00
Yaman Qalieh
7d31d32750 [core] Fix bugs in release helper and document (#2819) 2022-06-24 11:02:33 +02:00
Yaman Qalieh
abb4c17a0d bridges: Update maintainer (#2856) 2022-06-22 13:07:54 -04:00
Dag
a166899633 [FeedMerge] fix bugs (#2854)
* [FeedMerge] fix: sort items by timestamp descending

* [FeedMerge] fix: fetch 10 most recent items

This fixes a bug where the bridge e.g. fetched 10 items from the first feed and then nothing from the rest
2022-06-22 18:34:05 +02:00
Dag
7dc3449207 [KilledByGoogle] fix: broken enclosure url (#2852)
* [KilledByGoogle] fix: broken enclosure url

The previous enclosure url was HTTP 404.

Also add link to items.

* fix: close unclosed <a> tag
2022-06-22 18:33:48 +02:00
Dag
bde00447f1 [Telegram] fix: remove all enclosures except for videos (#2850) 2022-06-22 18:33:21 +02:00
Dag
9d5f59e2db [GoogleSearch] fix: improve bridge (#2848)
Sort properly by date.
Fix php errors.
Improve the date parsing logic.
Improve the content parsing by also including those with · as separator
2022-06-22 18:32:54 +02:00
Dag
e7aebb223d fix: catch everything (#2837)
Improve ux.
2022-06-22 18:32:22 +02:00
Dag
ee80f4918e refactor: action (#2836) 2022-06-22 18:30:37 +02:00
Dag
fad0dbb6ef refactor: fix exception handling (#2835)
* refactor: fix exception handling

The removed catch is never uses in php versions above 7.
The need for multiple catch statements like this is to support both php 5 and 7.

* remove traces of old exception handling

* add typehints

* dont treat exception code 0 specially
2022-06-22 18:30:06 +02:00
Dag
07927008eb refactor: CacheFactory (#2834) 2022-06-22 18:29:28 +02:00
Dag
b7b9378484 refactor: ActionFactory (#2833) 2022-06-22 18:28:07 +02:00
Dag
af5648d928 refactor: FormatFactory (#2832) 2022-06-22 18:27:20 +02:00
Austin Huang
e9b8a1f9f9 [Mastodon] Use ActivityPub outbox for Mastodon (et al.) feed (#2756)
* Use ActivityPub outbox for Mastodon (et al.) feed

closes #2754

* Better description for Mastodon bridge

I mean I could rename it to ActivityPub bridge if the maintainer so pleases

* [Mastodon] Please the lint

* [Mastodon] address feedback

* [Mastodon] fix link, address spelling case bug

* refactor

* [Mastodon] add username cache, fix try-catch, rename

* [Mastodon] shorten description to satisfy the lint

* [Mastodon] address feedback

* [Mastodon] support Secure Mode instances

* [Mastodon] add config documentation

* [Mastodon] update docs

Co-authored-by: Dag <me@dvikan.no>
2022-06-20 19:11:46 -04:00
Dag
8365a7a34d chore: bump required php version to 7.4 (#2838)
* chore: require min php 7.4

* Revert "feat: backport php 7.3 functions (#2803)"

This reverts commit 6df5a4bc14.

* [BandcampDaily] use array_key_first

* hard fail on php versions below 7.4

* update phpcompat linter
2022-06-19 21:45:33 +02:00
Joseph
192dc4dae2 [TikTokBridge] Add bridge (#2828) 2022-06-18 21:59:39 -04:00
Korytov Pavel
c78c1254a8 [NovayaGazetaEuropeBridge] Add bridge (#2827) 2022-06-18 21:57:40 -04:00
Yaman Qalieh
65e6d9f454 [XPathAbstract] Improve Media Url regex (#2845) 2022-06-19 02:19:32 +02:00
Gilles
ce63d8a706 [YGGTorrentBridge] Changed base URL and one parameter name (#2826) 2022-06-17 23:06:59 -04:00
Eugene Molotov
d0bea1627e [InstagramBridge] Fix incorrect cache timeout calculation (#2840)
It is expected that getCacheTimeout returns integer. In fact
it returned boolean value which lead to situation, where Instagram feeds
were not cached.
2022-06-18 03:02:31 +02:00
Dag
c6ba3e5280 [Parler] fix: use new api endpoint (#2831)
They also modified the json structure.
2022-06-17 21:18:29 +02:00
Jan Tojnar
10eb1c9a95 [FormatAbstract]: Ensure sanitizeHtml is given string (#2791)
Sometimes `Item::getContent` returns `null`, in which case `sanitizeHtml`
would pass it to `str_replace`, which would raise `E_DEPRECATED` on PHP 8.1.
2022-06-17 20:46:15 +02:00
Yaman Qalieh
9ac494b350 [AstrophysicsDataSystemBridge] Add bridge (#2796) 2022-06-17 20:00:31 +02:00
Joseph
c6100d95ca [UberNewsroomBridge] Add more regions & use region name from JSON (#2817) 2022-06-14 22:18:16 -04:00
Yaman Qalieh
33e3d9b596 [Kanali6Bridge] Add bridge (#2798) 2022-06-14 20:08:55 -04:00
Yaman Qalieh
9503f9ad7f [Release] 2022-06-14 (#2818) 2022-06-14 09:47:12 -04:00
Yaman Qalieh
3e2423d86b [MsnMondeBridge] Fix bridge (#2813) 2022-06-14 09:45:46 -04:00
Yaman Qalieh
90e0504da5 [Shimmie2Bridge] Fix bridge (#2814) 2022-06-14 09:45:35 -04:00
Yaman Qalieh
4b3b1ca163 [KununuBridge] Fix bridge for default parameters (#2816) 2022-06-14 09:45:01 -04:00
Yaman Qalieh
5b93bba1a3 [ZenodoBridge] Fix bridge (#2815) 2022-06-14 09:44:26 -04:00
Dag
b8786da137 Update 06_Public_Hosts.md (#2812)
Modify url to use https.
2022-06-10 17:19:45 +02:00
Kristian Salonen
166ead902d [Configuration.php] Update the version name to dev.2022-06-10 (#2811) 2022-06-10 15:05:34 +02:00
sal0max
de279de762 [CyanideAndHappiness] Add bridge (#2807) 2022-06-10 07:29:01 +02:00
Jan Tojnar
347f9a3eda [contents] Add MIME type for mp3 (#2809)
Without this, format tests fail on systems without `/etc/mime.types`.
2022-06-09 22:41:10 -04:00
Jan Tojnar
1af6cbeb1e [XML formats] Ensure elements are connected to DOM before further manipulation (#2806)
We are setting xmlns attributes at the root element but PHP would
still attach redundant ones to the DOM elements created with `createElementNS`.
That was because PHP reconciles namespace attributes when appending elements to DOM
but since we previously only attached the elements after all children were attached,
the reconciliation algorithm was not able to see the root element’s attributes.

To fix this, let’s attach each element to its parent immediately after it is created.
2022-06-09 18:33:23 +02:00
Yaman Qalieh
37f211a37e Add Laravel framework license 2022-06-09 12:13:07 -04:00
Yaman Qalieh
bea0595e5c Add php-urljoin license 2022-06-09 12:13:07 -04:00
Dag
6df5a4bc14 feat: backport php 7.3 functions (#2803)
* feat: backport php 7.3 functions

* fix: add license

* fix: formatting

* fix: add note in README regarding license
2022-06-09 18:00:51 +02:00
Yaman Qalieh
3927ecd822 [UsenixBridge] Add bridge (#2800) 2022-06-09 16:56:52 +02:00
pubak42
1b0a6f2813 [VixenBridge] New bridge (#2763) 2022-06-09 16:53:26 +02:00
Yaman Qalieh
8f0d90f653 [PixivBridge] Fix tags (#2799) 2022-06-08 23:05:56 -04:00
Yaman Qalieh
037d5866ca [BandcampDailyBridge] Fix list duplicates (#2795) 2022-06-08 19:37:06 -04:00
Yaman Qalieh
75c4c9f256 Revert "[Usenix] Add new bridge for USENIX (usenix.org) publications (#2772)" (#2793)
This reverts commit baa4ea8338.
2022-06-08 00:44:37 -04:00
Dag
baa4ea8338 [Usenix] Add new bridge for USENIX (usenix.org) publications (#2772)
Currently only supporting the ;login: publication.
2022-06-08 05:43:56 +02:00
Jan Tojnar
12ddee4054 tests/Formats: Simplify by using a base class (#2779)
There is a lot of redundancy. Let’s not repeat ourselves.

Unfortunately, since we do not install PHPUnit as a project dependency on CI,
it does not use the composer’s PSR-4 autoloader and the tests are unable to find
the `BaseFormatTest` class.

Until we resolve that, let’s load the class explicitly.
2022-06-08 02:17:32 +02:00
Dag
6582a66a2d Revert "Update tests.yml (#2788)" (#2792)
This reverts commit aa32040bd4.
2022-06-08 02:16:06 +02:00
sysadminstory
a4785370fa [DealabsBridge-HotUKDealsBridge-MydealsBridge-PepperBridgeAbstract] Fix (#2789)
the date handling

The deal posting date logic was wrong, and leaded to warnings and
notice. Now, only the feed with the deal sorted by date contains date
(the feed sorted by hottest deal does not contain a date anymore,
because there are no deal date in this case).
2022-06-07 23:55:15 +02:00
Dag
aa32040bd4 Update tests.yml (#2788) 2022-06-07 23:33:16 +02:00
Jan Tojnar
44e8007d9c tests: Use PSR-4-style namespaces (#2778)
We cannot yet switch to namespaces for RSS-Bridge itself but for tests we are not limited by BC.
It does not actually do anything since PHPUnit will search for the test files without the help of the autoloader but it still makes the directory cleaner.
2022-06-07 23:22:33 +02:00
Jan Tojnar
90d22f0d80 [{Atom,Mrss}Format]: Generate using DomDocument (#2771)
* [AtomFormat]: Generate using DomDocument

This will escape the HTML content for us as needed.

* [MrssFormat]: Generate using DomDocument

This will escape the HTML content for us as needed.
2022-06-07 23:22:03 +02:00
Jan Tojnar
fb501652d5 Formats: Remove display & related method (#2776)
Format should not be responsible for sending HTTP response.
2022-06-07 18:05:33 +02:00
Joseph
e85932b1a5 [BrutBridge] Fix bridge (#2787) 2022-06-07 18:05:03 +02:00
Korytov Pavel
53f9970403 [EuronewsBridge] Add bridge (#2786) 2022-06-07 10:25:20 -04:00
Jan Tojnar
19ad2584da [NFLRUSBridge] Remove byte-order-mark (#2777)
With UTF-8 byte-order mark in the file, the `ListActionTest::testOutput`
would fail after converting tests to PSR-4 namespaces:

    invalid JSON output: Syntax error
    Failed asserting that null is not null.

This is because ListAction::execute tries to create the bridge objects
and, when the files containing the bridge classes are not loaded yet,
the autoloader starts including them. Since this happens after output
buffering has begun, any text in the PHP file before the `<?php` tag
such as the BOM will end up in the buffer to be parsed by `json_decode`.

Previously, it worked by chance thanks to some other test including the file
before `ListActionTest`. With the restructuring, `Actions\ListActionTest`
will run sooner and become responsible for triggering the autoloader.

To prevent this in the future, I also disallowed BOM in the coding style.
2022-06-07 04:59:22 +02:00
somini
190c95fa62 [PCGWNewsBridge]: New Bridge (#2785) 2022-06-06 01:02:15 +02:00
Dag
678e5d9866 [NeuviemeArt] Exterminate dead bridge (#2784)
They moved to https://www.bubblebd.com/9emeart

Fixes #2774
2022-06-05 22:56:54 +02:00
Yaman Qalieh
4787eb3799 [WordPressMadaraBridge] Add Bridge (#2782) 2022-06-05 14:40:43 -04:00
Yaman Qalieh
a863234474 [MangaDexBridge] add chapter search context (#2783) 2022-06-05 14:28:05 -04:00
Korytov Pavel
4260be26a2 [EconomistWorldInBriefBridge] indent with tabs instead of spaces (#2781) 2022-06-05 18:40:48 +02:00
Mynacol
713d06ba08 [GitlabIssueBridge] Code cleanup (#2780)
- Rename parseMRDescription() -> parseMergeRequestDescription()
- Move parseMergeRequestDescription() below parseIssueDescription()
- Inline getProjectURI()
2022-06-05 18:39:54 +02:00
Korytov Pavel
7256d1138b [EconomistWorldInBriefBridge] Add bridge (#2765) 2022-06-05 17:16:11 +02:00
Dag
71310d2c5a [OsmAndBlog] Remove bridge (#2775)
They revamped their page. The feed has been returning a single item
for some time.

Their blog can be followed at:
https://osmand.net/blog/atom.xml
2022-06-05 03:51:37 +02:00
Dag
92d813fbea [NotAlways] fix: broken url (#2773)
The /all url now actually points to a specific item.
I think we want the frontpage for this.

Fixes:
Fatal error: Uncaught Error: Call to a member function find() on null in NotAlwaysBridge.php:37
2022-06-05 02:56:51 +02:00
Mynacol
3f896f9465 [GitlabIssueBridge] Add bridge (#2760)
* [GitlabIssueBridge] new bridge

This tracks issue comments on arbitrary gitlab projects.

* [GitlabIssueBridge] Prepare for Merge Request support + fixes

- Proper UIDs
- Default bridge name fixed
- Fix cache identifiers
- Add TODOs

* [GitlabIssueBridge] creation timestamp preferred

And prefer original author over editor.

* [GitlabIssueBridge] Do not add date to item title

Prettier without it.

* [GitlabIssueBridge] Support Merge Requests

This bridge can now generate feeds for Merge Requests.

* [GitlabIssueBridge] typo

* [GitlabIssueBridge] Fix Img src attr in comments

* [GitlabIssueBridge] Fix function call

* [GitlabIssueBridge] Fix test

Use gitlab.com if no h parameter was given.
Fixes a phpunit test.

* [GitlabIssueBridge] linting

* [GitlabIssueBridge] Add MR support to description

* [GitlabIssueBridge] Move function collectData

* [GitlabIssueBridge] rm single-use class constants

* [GitlabIssueBridge] Remove manual caching

Just depend on rss-bridges built-in caching.
2022-06-04 23:59:10 +02:00
Shikiryu
b7e1dc1ab1 [KhinsiderBridge] fix RSS because of the new layout (#2767)
* [KhinsiderBridge] fix RSS because of the new layout

* [KhinsiderBridge] fix phpcs
2022-06-04 22:41:37 +02:00
sysadminstory
8e41887393 [DealabsBridge-HotUKDealsBridge-MydealsBridge] Fix example values (#2766)
Added real example values for discussion to allow automatic testing.

Updated keywords example value to be sure there will be some results.
2022-06-04 22:40:20 +02:00
Mynacol
8865521b3b [GolemBridge] Remove image galleries (#2761)
Do not add all images of the image gallery, but only the preselected one.

Often, the same gallery is used multiple times with different preselected
images. The previous implementation always added all images of the
gallery, cluttering the article. This patch only adds the preselected one.

The no-js link wrapping around the gallery leads to a 403 Forbidden
page, so linking that doesn't work to really support galleries.
2022-06-04 22:27:24 +02:00
Austin Huang
8172d10bb5 [Amazon & AmazonPriceTracker] Add Turkey, close #2665 (#2758)
* [AmazonPriceTracker] Add Turkey, close #2665

* [Amazon] Add Sweden & Turkey

consistent with price tracker
2022-06-04 21:59:52 +02:00
Park0
299ad87168 [Marktplaats] #2553 example values added (#2752)
For automation tests example values are needed
2022-06-04 21:06:38 +02:00
Dag
d60d8313d0 fix: type error in function call (#2769)
Fixes:
Argument 2 passed to getContents() must be of the type array, int given
2022-06-04 21:05:43 +02:00
Yaman Qalieh
1fd2f37bb4 [PixivBridge] Fix 404 for fullsize novel images (#2751) 2022-06-04 20:53:10 +02:00
Dag
04b1609ce0 docs: refactor table of public instances (#2749) 2022-06-04 20:52:10 +02:00
Christian Schabesberger
2fa24e780b Fix nordbayern (#2730) 2022-06-04 20:50:16 +02:00
quickwick
3b04e318ae [SlusheBridge] New bridge (#2700) 2022-06-04 20:10:07 +02:00
Alexandre Alapetite
05cd1c0b67 [Core] Add expose to dockerfile (#2762) 2022-05-30 20:05:42 +02:00
Dag
cb05cacd6a fix: add 429 to status codes (#2757) 2022-05-27 15:25:12 +02:00
Bocki
4d18312604 [Core] Prtester fix list fix (#2753) 2022-05-25 20:26:39 +02:00
Joseph
85e5ce2679 [UberNewsroomBridge] Add bridge (#2748) 2022-05-25 09:43:18 +02:00
Bocki
7afc577e97 [core] Fix nested selection lists (#2750) 2022-05-25 09:38:53 +02:00
KamaleiZestri
462319344b [CubariBridge] New Bridge (#2747) 2022-05-24 13:34:40 +02:00
Dag
5cc34b884a [core] Improve getContents docs (#2742) 2022-05-22 21:27:23 -04:00
KamaleiZestri
dd025894e9 [PillowfortBridge] Modify example value for Pillowfort Bridge (#2746) 2022-05-22 15:30:45 -04:00
Dag
1d0a0b927b fix: use accept header when fetching feed (#2737)
* fix: use accept header when fetching feed

* fix: include atom too, and reuse constants from format classes

* add a catch all accept header
2022-05-18 00:18:33 +02:00
Dag
4007afdcf5 Merge branch 'autoloading' into master 2022-05-17 23:59:18 +02:00
Dag
7d00b0c5df fix: include http code in exception (#2726) 2022-05-17 23:47:12 +02:00
Dag
0212c4790f fix: connectivityaction (#2725) 2022-05-17 23:46:37 +02:00
Kingsley Yung
7a87a09fc5 [YouTubeCommunityTab] Fix error occuring when YouTube returns non-English webpage. (#2739) 2022-05-17 09:35:16 +02:00
sysadminstory
1e3f5f3ad3 [PepperBridge] Update CSS selectors (#2740)
Updated some CSS selectors to follow the website change
2022-05-17 09:34:03 +02:00
Yaman Qalieh
f709778b28 [MydealsBridge] Fix Example value (#2728) 2022-05-14 08:04:21 -04:00
Yaman Qalieh
f4a0711b62 docker: fix find error (#2733) 2022-05-14 07:18:58 -04:00
Dag
4d069fcf99 remove unnecessary includes 2022-05-13 09:35:25 +02:00
Dag
f00f90328d refactor: extract class PepperBridgeAbstract 2022-05-13 09:29:56 +02:00
Yaman Qalieh
bb6d553dd5 Revert "refactor: remove unnecesary includes" (#2723)
This reverts commit fd449be4eb.
2022-05-12 16:28:03 -04:00
Joseph
97b513823d [HaveIBeenPwnedBridge] Fix item URIs (#2724) 2022-05-12 22:16:34 +02:00
Eric G
e01f0bcaf2 [GiteaBridge] Rewrite to decouple from Gogs and add contexts (#2718) 2022-05-12 22:15:03 +02:00
Yaman Qalieh
e5829d37b6 [HaveIBeenPwnedBridge] Use API to get Data (#2720) 2022-05-12 21:53:03 +02:00
Yaman Qalieh
73b1a6a7aa [FDroidRepoBridge] Add F-Droid Repo Bridge 2022-05-12 09:37:11 -04:00
Yaman Qalieh
e07fac777a core: Enable zip extension 2022-05-12 09:37:11 -04:00
Dag
fd449be4eb refactor: remove unnecesary includes 2022-05-12 15:15:09 +02:00
Dag
829fc6cca2 docker: Switch to nginx in docker image (#2721)
Co-authored-by: Yaman Qalieh <ybq987@gmail.com>
2022-05-11 20:19:25 -04:00
Dag
fcc3707210 refactor: swap the order of sprintf values 2022-05-11 22:37:59 +02:00
Dag
d5e9dbf47d refactor: restore some useful comments 2022-05-11 22:35:03 +02:00
User123698745
96a63a8e81 [PicukiBridge] fix image not displaying (#2717) 2022-05-10 19:53:29 +02:00
quickwick
9110b70f07 [TwitterV2Bridge] Properly include quoted tweets (#2713) 2022-05-10 09:41:12 +02:00
Yaman Qalieh
6547ed0c04 [docs] Add documentation for html.php functions (#2714) 2022-05-10 09:37:53 +02:00
Dag
8982995445 refactor: remove unused method 2022-05-09 23:32:45 +02:00
Dag
76084cdcca fix: logic bug in limiting 2022-05-09 21:02:21 +02:00
Dag
a28dca2c9d chore: upgrade phpunit 7 => 9
Upgraded with:

composer require -W --dev phpunit/phpunit:^9
2022-05-09 20:52:02 +02:00
sysadminstory
51f0d046d0 [AllocineFRBridge] Automatically find the last season for every show (#2709)
The bridge now finds the last season URI by itself, and don't rely on
static URL stored in the bridge itself.
2022-05-08 17:22:39 +02:00
Christian Schabesberger
fb2ed95368 Fix nordbayern (#2708)
* fix newspaper thumbnails are shown again

* show article teaser on top of title image for NN
2022-05-08 16:37:53 +02:00
Mynacol
36d11fd06e [XenForoBridge] Fix error if message is < 70 chars (#2707)
At the time of writing, this occurs on the following thread:
https://forum.xda-developers.com/t/optimized-lineageos19-1-v4-0-23apr.4426575/

Fixes the following error:
ValueError: strpos(): Argument #3 ($offset) must be contained in argument #1 ($haystack) in ./rss-bridge/bridges/XenForoBridge.php:272
Stack trace:
0 ./rss-bridge/bridges/XenForoBridge.php(272): strpos()
1 ./rss-bridge/bridges/XenForoBridge.php(146): XenForoBridge->extractThreadPostsV2()
2 ./rss-bridge/actions/DisplayAction.php(134): XenForoBridge->collectData()
3 ./rss-bridge/index.php(24): DisplayAction->execute()
4 {main}
2022-05-08 16:25:01 +02:00
quickwick
d107592094 Don't hide quoting tweets when 'hide retweets' is selected (#2706) 2022-05-08 16:22:31 +02:00
Yaman Qalieh
0ce71d561d [PixivBridge] [UnsplashBridge] Fix deprecated null (#2705) 2022-05-08 16:17:26 +02:00
Dag
f5a51038cc fix: error when passing null values
This bug was introduced by me when refactoring the http client.

Fixes:

Fatal error: Uncaught TypeError: Argument 2 passed to getContents() must be of the type array, null given
2022-05-08 04:42:24 +02:00
Yaman Qalieh
3476b06ee0 [MangaDexBridge] Exclude external chapters (#2703) 2022-05-08 04:22:33 +02:00
Yaman Qalieh
158ee41be4 [AtomFormat] Remove redundant fallback content (#2702) 2022-05-08 04:21:32 +02:00
sysadminstory
37843e8777 [RadioMelodieBridge] Fix date parsing (#2701)
The date is now correctly parsed for every month in the year (There are months mane in french that are 3 letters long
2022-05-08 04:19:06 +02:00
Mynacol
56e991122b [GolemBridge] Add golem.de bridge (#2696) 2022-05-08 04:08:55 +02:00
Dag
5d77d14f9d feat: add retry logic to the http client (#2692)
* refactor: extract http client

* feat: add retry logic to http client
2022-05-08 03:58:57 +02:00
Dag
0c7a7f320f refactor: BridgeFactory (#2691) 2022-05-08 03:58:42 +02:00
Dag
b2f1d051fc fix: don't bork upstream with http status code -1 (#2690) 2022-05-08 03:57:46 +02:00
Dag
641e2eedf5 test: exclude Pixiv for a particular test 2022-05-08 03:55:24 +02:00
Binnette
bc773a49f8 Full rewrite of bridge DeveloppezDotCom (#2689) 2022-05-08 03:38:33 +02:00
Yaman Qalieh
410daee1d5 [PixivBridge] Add User context (#2650) 2022-05-08 02:46:57 +02:00
Christian Schabesberger
adeaede930 [NordbayernBridge] Fix Bridge (#2675) 2022-05-02 19:06:30 +02:00
Dag
9b82ff352d fix: Fatal error: Uncaught ArgumentCountError 2022-05-01 21:35:52 +02:00
Nemo
31455b6838 [npci] Adds new NPCI Bridge (#2651) 2022-04-29 00:01:18 +02:00
Dag
63b08f7da9 Update app.json 2022-04-26 22:43:13 +02:00
Dag
61cfbe6c53 Update app.json 2022-04-26 22:41:34 +02:00
quickwick
4c26950b71 [TwitterV2Bridge] Fix empty object check (#2673) 2022-04-26 12:11:26 +02:00
pirnz
9dc31dfcfa [AsahiShimbunAJWBridge] Updated Asia section links (#2671) 2022-04-26 12:10:10 +02:00
Dag
db8462e6fa chore: add scripts section to composer.json (#2684) 2022-04-26 01:59:50 +02:00
Dag
19a8165fc6 docs: add host to public instances (#2685) 2022-04-26 01:41:04 +02:00
Dag
0ef298f9cc refactor: add php autoloader (#2655) 2022-04-26 00:57:59 +02:00
Corentin Garcia
b090b17bbf [RobinhoodSnacksBridge] fix bridge (#2676) 2022-04-26 00:53:18 +02:00
sysadminstory
ca749e7bad [ZoneTelechargement] Remove bridge (#2678)
Website announced the shutdown
2022-04-25 20:01:39 +02:00
Patrick Collins
e1c898848f [contents.php] Fix incorrect reference to UnexpectedResponseException's responseBody. (#2677) 2022-04-23 10:04:56 +05:00
Eric G
46a356b0b2 [GogsBridge] Add protocol to examplevalue (#2668) 2022-04-18 22:31:26 +02:00
Joseph
fe042305e4 [GoogleSearchBridge] Update bridge (#1869) 2022-04-16 23:16:38 +02:00
Loïc Fürhoff
669e92357a [Arte7Bridge] Exclude trailers and sort by v2 (#2664) 2022-04-16 23:08:27 +02:00
Nemo
1dec457b7b docu: Add back rss-bridge.bb8.fun (#2666) 2022-04-16 23:05:26 +02:00
TotalCaesar659
a38951b911 general: Update URLs to HTTPS (#2667) 2022-04-16 23:03:15 +02:00
Dag
b11f1368bf Update 06_Public_Hosts.md 2022-04-15 22:51:30 +02:00
Dag
1a698b3554 fix: remove dead public instances
https://rss-bridge.bb8.fun 404
https://myrss4fun.xyz not serving rss-bridge
https://rssbridge.fossdaily.xyz Curl failed for "https://rssbridge.fossdaily.xyz": Could not resolve host: rssbridge.fossdaily.xyz (6)
https://bridge.noisebridge.info 502
https://rss-bridge.esmailelbob.xyz 502
2022-04-15 22:51:12 +02:00
Dag
73ebdbf67a Revert "[Arte7Bridge] Exclude trailers and sort by (#2660)" (#2662)
This reverts commit 924eaf2011.

That commit broke the bridge.
2022-04-15 22:02:41 +02:00
Eugene Molotov
ac766aa47f [RutubeBridge] Add bridge (#2661) 2022-04-16 00:37:38 +05:00
Nemo
2be613e015 [BookMyShowBridge] Add new bridge (#1349) 2022-04-15 19:55:32 +02:00
Loïc Fürhoff
924eaf2011 [Arte7Bridge] Exclude trailers and sort by (#2660) 2022-04-14 23:20:09 +02:00
Alex Balgavy
d082bfca4a [SeznamZpravyBridge] fix: broken bridge (#2658) 2022-04-14 14:38:16 +02:00
Dag
91283f3a62 fix: deprecation notice (#2656) 2022-04-13 21:35:54 +02:00
Dag
d6beb713b5 chore: upgrade dependencies and improve package.json (#2648)
* chore: update composer dependencies

$ composer update
Loading composer repositories with package information
Updating dependencies
Info from https://repo.packagist.org: #StandWithUkraine
Lock file operations: 0 installs, 8 updates, 0 removals
  - Upgrading doctrine/instantiator (1.4.0 => 1.4.1)
  - Upgrading myclabs/deep-copy (1.10.2 => 1.11.0)
  - Upgrading phpdocumentor/reflection-docblock (5.2.2 => 5.3.0)
  - Upgrading phpdocumentor/type-resolver (1.4.0 => 1.6.1)
  - Upgrading phpspec/prophecy (1.13.0 => v1.15.0)
  - Upgrading phpunit/php-file-iterator (2.0.4 => 2.0.5)
  - Upgrading sebastian/exporter (3.1.3 => 3.1.4)
  - Upgrading symfony/polyfill-ctype (v1.23.0 => v1.25.0)
Writing lock file
Installing dependencies from lock file (including require-dev)
Package operations: 0 installs, 8 updates, 0 removals
  - Upgrading symfony/polyfill-ctype (v1.23.0 => v1.25.0): Extracting archive
  - Upgrading phpdocumentor/type-resolver (1.4.0 => 1.6.1): Extracting archive
  - Upgrading phpdocumentor/reflection-docblock (5.2.2 => 5.3.0): Extracting archive
  - Upgrading sebastian/exporter (3.1.3 => 3.1.4): Extracting archive
  - Upgrading phpunit/php-file-iterator (2.0.4 => 2.0.5): Extracting archive
  - Upgrading doctrine/instantiator (1.4.0 => 1.4.1): Extracting archive
  - Upgrading phpspec/prophecy (1.13.0 => v1.15.0): Extracting archive
  - Upgrading myclabs/deep-copy (1.10.2 => 1.11.0): Extracting archive
Package phpunit/php-token-stream is abandoned, you should avoid using it. No replacement was suggested.
Generating autoload files
16 packages you are using are looking for funding.
Use the `composer fund` command to find out more!

* chore: add dev-dependency squizlabs/php_codesniffer (phpcs)

$ composer require --dev squizlabs/php_codesniffer
Info from https://repo.packagist.org: #StandWithUkraine
Using version ^3.6 for squizlabs/php_codesniffer
./composer.json has been updated
Running composer update squizlabs/php_codesniffer
Loading composer repositories with package information
Updating dependencies
Info from https://repo.packagist.org: #StandWithUkraine
Lock file operations: 1 install, 0 updates, 0 removals
  - Locking squizlabs/php_codesniffer (3.6.2)
Writing lock file
Installing dependencies from lock file (including require-dev)
Package operations: 1 install, 0 updates, 0 removals
  - Installing squizlabs/php_codesniffer (3.6.2): Extracting archive
Package phpunit/php-token-stream is abandoned, you should avoid using it. No replacement was suggested.
Generating autoload files
16 packages you are using are looking for funding.
Use the `composer fund` command to find out more!

* chore: add package type => "project" in composer.json
2022-04-13 21:04:27 +02:00
Dag
d62b977394 refactor: ./tests (#2649)
* refactor: ./tests

* test: consolidate testsuites

* refactor: move config setup into rssbridge.php

Makes it easier to unit test.

* lint
2022-04-13 21:04:10 +02:00
Austin Huang
183004f954 Update 06_Public_Hosts.md (#2654)
1. nixnet.xyz => nixnet.services
2. Add my instance

Supersedes #2653
2022-04-13 20:36:28 +02:00
Shikiryu
ff8ece213f [PicalaBridge] Add new bridge (#2646) 2022-04-13 13:25:32 +02:00
Nemo
3e5675c256 [GoodreadsBridge] Add new bridge (#1559) 2022-04-13 13:18:05 +02:00
Dag
5a7d305e07 [Nordbayern] fix: problem with absolute and relative link (#2637)
* [Nordbayern] fix: problem with absolute and relative link

Fixes:

cURL error: Could not resolve host: www.nordbayern.dehttps
2022-04-12 23:40:37 +02:00
Dag
7379e2b3d5 [Parler] feat: add new bridge (#2634) 2022-04-12 23:39:32 +02:00
Dag
57c8806954 [ParuVenduImmo] fix: try to repair broken css selectors (#2641)
* [ParuVenduImmo] fix: try to repair broken css selectors

Needs more work.
2022-04-12 23:37:54 +02:00
Dag
b6e8350596 fix: a bunch of small changes in multiple bridges (#2644)
Mostly refactors.
2022-04-12 23:37:30 +02:00
Dag
5b7dd45b20 [UsbekEtRica] fix: broken css selectors (#2643) 2022-04-12 23:37:17 +02:00
Dag
f9801a5c58 [RoadAndTrack] fix: broken css selectors (#2642) 2022-04-12 23:37:05 +02:00
Dag
563c099d80 [NFLRUS] fix: broken css selectors (#2640)
This bridge needs more work.
2022-04-12 23:35:04 +02:00
Dag
b6e8e3ea6e [N26] fix: broken css selectors (#2639) 2022-04-12 23:34:52 +02:00
Dag
9e2e32a19d [Amazon] fix: broken css selectors (#2638) 2022-04-12 23:34:40 +02:00
Dag
df5c259375 [LaCentrale] fix: broken css selectors (#2636) 2022-04-12 23:34:23 +02:00
quickwick
6021d2ffa6 [GelbooruBridge] Change tag examplevalue to one valid for all inheriting bridges (#2645) 2022-04-12 16:52:34 +02:00
Dag
908da78113 Update phpcs.xml 2022-04-11 02:28:41 +02:00
Dag
a28481aaa8 [XenForo] fix: sort items by date in descending order (#2633) 2022-04-11 00:42:53 +02:00
Dag
bb81af086f [Castorus] fix: htmlentities bug (#2632) 2022-04-10 22:47:02 +02:00
Dag
60f1c46779 docs: move screenshots from wiki to repo (#2631) 2022-04-10 22:23:06 +02:00
Corentin Garcia
ae760e40cc replace wiki links with documentation links (#2630)
* docs: update composer support links

* docs: update link from wiki to docs for github issue template

* docs: update link pointing to wiki to point to new documentation

* docs: replace wiki links by documentation links in README and
CONTRIBUTING files
2022-04-10 21:58:29 +02:00
dag
5a733b3d82 feat: add limit options to the slowest bridges 2022-04-10 18:56:24 +02:00
dag
0b40f51c01 [Picuki] fix: item parsing (#2619)
Fixes a problem with the entire content being a link.

Also truncate title.

They have referrer checks on their images. So clicking the
enclosure doesnt work. Will fix later.
2022-04-10 18:54:48 +02:00
dag
dbee47f1d6 fix: give better error message when feed can't be parsed (#2618) 2022-04-10 18:54:32 +02:00
dag
c3a106892d fix: require curl extension (#2617) 2022-04-10 18:54:18 +02:00
quickwick
db28bedb23 [TwitterV2Bridge] Changes to output HTML/CSS, larger display image by default (#2626) 2022-04-10 18:53:35 +02:00
User123698745
aacf5812ff [GiphyBridge] include search text in feed name (#2628)
Co-authored-by: User123698745 <usr123698745+git@gmail.com>
2022-04-10 18:06:33 +02:00
User123698745
bf2f9a06f9 [Docker] force unix line ending on bash files to prevent docker run failing on windows (#2629) 2022-04-10 18:05:43 +02:00
User123698745
7833d0e6c3 [GiphyBridge] include bundle parameter in api calls to reduce bandwidth (#2627) 2022-04-10 15:21:43 +02:00
Corentin Garcia
c498749c2b [TwitterEngineeringBridge] add bridge #2385 (#2623) 2022-04-10 11:47:42 +02:00
User123698745
722f9ff0ce [GiphyBridge] use not rate limited public api key (#2625)
Co-authored-by: User123698745 <usr123698745+git@gmail.com>
2022-04-10 11:37:02 +02:00
Vít Kabele
5c08984714 InstagramBridge.php: Display usernames and hashtags as links. (#1582) 2022-04-08 23:47:10 +02:00
dag
dc01891634 fix: enclosure link privacy (#2620) 2022-04-08 23:14:43 +02:00
dag
cce11964a4 feat: add a timeout option for http client (#2600) 2022-04-08 21:22:13 +02:00
Corentin Garcia
8c18c02c65 [GatesNotesBridge] Add feedaxpander bridge for Bill Gate's blog (fix issue #2386) (#2611) 2022-04-08 21:21:13 +02:00
Antoine Turmel
51d27300be [FeedMergeBridge] Add new bridge (#1385)
* [FeedMergeBridge] Add new bridge

Here is a bridge that merges two or more feeds into one.

Co-authored-by: Bocki <henning@bocklage.com>
Co-authored-by: Dag <me@dvikan.no>
2022-04-08 21:13:05 +02:00
quickwick
c0e2a430ab [TwitterV2Bridge] Add parameter to include only media tweets (#2614) 2022-04-07 09:00:28 +02:00
quickwick
daae089299 [TwitterV2Bridge] Changes to parameters and output titles (#2612) 2022-04-06 20:56:56 +02:00
Joseph
d98add2cac [TelegramBridge] Fix issues & add support for location messages (#2133) 2022-04-06 10:15:21 +02:00
Tobias Alexander Franke
a3b0b91dee [BinanceBridge] Remove announcements because of Cloudflare issue (#2610) 2022-04-05 23:20:27 +02:00
langfingaz
6ffe531e4f [UnsplashBridge] extend functionality (#1813) 2022-04-05 15:00:10 +02:00
Bocki
fb28107cc4 [Core] Fix prtester context issue (#2609) 2022-04-05 14:46:42 +02:00
Bocki
2c50bbae95 [AssociatedPressNewsBridge] fix checks (#2608) 2022-04-05 14:37:15 +02:00
Joseph
8f9314947b [AssociatedPressNewsBridge] Add bridge (#1475) 2022-04-05 14:03:25 +02:00
Mikalai Daronin
b24cdd47f0 [AlfaBankByBridge] new bridge for alfabank.by (#2349) 2022-04-05 13:14:09 +02:00
Tomer Shvueli
233a3cb643 general: Added a button to install RSS Bridge on Cloudron (#2559) 2022-04-05 12:22:47 +02:00
Bocki
91c6645fc7 [core] fix testing changes (#2607) 2022-04-05 12:17:19 +02:00
quickwick
780581939a [TwitterV2Bridge] New Bridge for Twitter v2 API (#2471)
* New Bridge for Twitter using v2 API

* Top comment block, tweaks to match contributing guide

* [TwitterV2Bridge] new Bridge (sort of)

* Discovered the point of, and re-added, no image scaling option

* Fix the phpcs sniff violations (I hope)

* More linter fixes, I figured out how to use phpcs locally

* Removed unnecessary custom version of getContents function

* Limit query to 100 tweets, valid example query, improved error handling

* Added config doc (correctly, I hope) with link from DESCRIPTION

* little tweak to doc
2022-04-04 21:13:05 +02:00
arnd-s
0d305f1530 [TwitterBridge] Migration to API V1.1 (#2433) 2022-04-04 19:50:59 +02:00
Nemo
e1e9a12440 [AmazonPriceTrackerBridge] Minor fix for parser, and new strategy (#2603) 2022-04-04 19:41:40 +02:00
Bocki
d34b94848b [Core] Adapt list behavior (#2605) 2022-04-04 19:40:46 +02:00
Nemo
2eaf48de99 Fix AppleAppStoreBridge (#2604) 2022-04-04 19:05:52 +02:00
LogMANOriginal
d3bb00f754 docs: Explain loadCacheValue and saveCacheValue
This adds documentation for methods added via #1380.
2022-04-03 12:19:13 +02:00
dag
00a3f80ac4 [Mangareader] chore: remove dead bridge (#2597)
It's currently timing out.
2022-04-03 10:26:17 +02:00
dag
260fc41d72 [RTFB] chore: remove dead and unmaintained bridge (#2596) 2022-04-03 10:23:06 +02:00
quickwick
28f5066fc4 Delete broken, unneeded bridges (#2595) 2022-04-03 10:10:56 +02:00
Michael Bemmerl
aa83a990d1 [OtrkeyFinderBridge] Remove HTML in title (#2594)
* [OtrkeyFinderBridge] Provide a better example that actually returns results.

* [OtrkeyFinderBridge] Remove HTML in filename.
2022-04-03 10:09:42 +02:00
Yaman Qalieh
7dcf09a876 [GitHub] Allow custom search query (#2593) 2022-04-03 10:07:35 +02:00
somini
d123e6007e Fixup deprecations on PHP 8 (#2592)
* Fixup deprecations on PHP 8

Fix #2448

* Configure a default fallback for getInput function

* Appease phpcs

* Avoid changing getInput function

Revert "Configure a default fallback for getInput function"

This reverts commit 94004c5104.
2022-04-03 09:53:13 +02:00
quickwick
a5eb02d3c3 [MixcloudBridge] switch to using API (#2591)
* switch to using public API

* switch to different API endpoints

* fix: urlencode username

Co-authored-by: Dag <me@dvikan.no>
2022-04-03 09:51:41 +02:00
dag
7b168a29f0 [WordPressPluginUpdate] fix: broken bridge (#2572)
I think they removed the changelog html page. Or maybe it
was a redirect. Anyways, this change uses their json api
to fetch plugin data.
2022-04-03 09:38:34 +02:00
dag
bed20e9f28 feat: extract curl ua to config value (#2568)
* exclude config.default.ini.php from phpcs
2022-04-03 09:37:39 +02:00
Joseph
42788cd3ee [YahtzeeDevDiaryBridge] Remove bridge (#2580)
Website has rss feeds. https://www.escapistmagazine.com/category/yahtzees-dev-diary/feed/
2022-04-02 21:11:34 +02:00
Yaman Qalieh
fb0e7ede89 [ParksOnTheAirBridge] Fix links (#2590) 2022-04-02 12:53:10 +02:00
LogMANOriginal
f311fb8083 [BridgeAbstract] Add loadCacheValue() and saveCacheValue() (#1380)
* [BridgeAbstract] Add loadCacheValue() and saveCacheValue()

Bridges currently need to implement value caching manually, which
results in duplicate code and more complex bridges.

This commit adds two protected functions to BridgeAbstract that make
it possible for bridges to store and retrieve values from a temporary
cache by key.

Co-Authored-By: Roliga <roliga.here@gmail.com>

Co-authored-by: Roliga <roliga.here@gmail.com>
2022-04-02 08:15:28 +02:00
Jacob Zelek
40a4e7b7c2 [ParksOnTheAir] New bridge for amateur radio (#2086) 2022-04-01 20:17:00 +02:00
Yaman Qalieh
73cc791ce1 [MangaDexBridge] Add new bridge (#2583) 2022-04-01 20:15:47 +02:00
Yaman Qalieh
d4707fc119 [CraigslistBridge] Fix notice with nearby results (#2588)
If the search query includes searchNearby=1, nearby results do not have
.result-hood to indicate location, instead using .nearby.
2022-04-01 16:38:37 +02:00
Yaman Qalieh
8aa091beda [GithubIssueBridge] Fix notice with reviews (#2589)
Some timeline items, like review threads and the first comment on PRs,
have no header, so this handles the first comment and adds a generic
title if that doesn't work.
2022-04-01 16:38:07 +02:00
Foxocube
d6695c0e73 [FurAffinityUserBridge] Replate username/password with cookie login (#1641) 2022-03-31 20:28:46 +02:00
Bocki
b6798b9878 general: doc fix (#2586) 2022-03-31 20:27:36 +02:00
Bocki
6baf38f251 general: fix doc (#2585) 2022-03-31 20:17:40 +02:00
Bocki
e6ae91b4d0 [FuraffinityuserBridge] Add doc about login (#2584) 2022-03-31 20:13:19 +02:00
Joseph
e525b5b427 [OpenClassroomsBridge] Remove bridge (#2582) 2022-03-31 19:41:59 +02:00
dag
983df45264 [CourrierInternationalBridge] fix: don't break on unusual feed items #2570 (#2571)
* [CourrierInternationalBridge] fix: skip unusual feed items #2570

This skips feed items who don't have content.
The one I encountered was a horoscope.
This change makes sure the bridge dont errors out.
2022-03-31 17:01:11 +02:00
dag
8717c33646 [Glassdoor] fix: repair broken bridge (#2577) 2022-03-31 17:00:14 +02:00
Joseph
7280ed7df7 [ScribdBridge] Update example profile URL value (#2578) 2022-03-31 15:58:23 +02:00
Joseph
d6b431a34b [DownDetectorBridge] Remove bridge (#2579) 2022-03-31 15:33:33 +02:00
Teemu Ikonen
aa0aa727ad [Arte7Bridge] Support all languages (#2543) 2022-03-31 11:17:07 +02:00
dag
06ef3946cd [PokemonTV] fix: use exampleValue that returns items (#2573) 2022-03-31 09:55:55 +02:00
dag
e94d447727 [DaveRamseyBlogBridge] fix: remove dead bridge #2345 (#2574) 2022-03-31 09:52:28 +02:00
dag
25e9f69261 [ElsevierBridge] fix: broken bridge (#2575) 2022-03-31 09:49:30 +02:00
dag
3e363bbc20 [FootitoBridge] chore: remove bridge (#2576) 2022-03-31 09:46:04 +02:00
Matt DeMoss
cf2dad3ab8 Reducer (retrying after failed tests) (#2273) 2022-03-30 01:50:07 +02:00
floviolleau
d6a4f2fd5b [VieDeMerdeBridge] fix due to website changes (#2567) 2022-03-30 00:58:29 +02:00
Yaman Qalieh
d27c1a99c2 [YeggiBridge] Add model source and tags (#2566) 2022-03-30 00:57:25 +02:00
Yaman Qalieh
0d80f2d5c3 [YeggiBridge] Extend description for discovery (#2565) 2022-03-29 23:48:28 +02:00
Joseph
a485beadd7 [FlickrBridge] Add content option to By username (#1861) 2022-03-29 23:46:55 +02:00
dag
ec7d2a4afb [QPlayBridge] chore: remove dead bridge (#2564) 2022-03-29 23:33:46 +02:00
dag
427becf441 [ThingiverseBridge] chore: remove dead bridge (#2563) 2022-03-29 23:13:14 +02:00
dag
267fdb27fc chore: remove dead bridge (#2562) 2022-03-29 23:12:47 +02:00
Bocki
ac242609f4 [core] Update simplehtmldom to latest released (#2556) 2022-03-29 22:45:26 +02:00
dag
461269195b fix: ignore partial json_encode() errors in JsonFormat (#2554)
Without this change, JsonFormat simply returns
an empty array. #2283
2022-03-29 22:45:00 +02:00
dag
060b4c7d58 [AnimeUltimeBridge] fix: convert strings from iso-8859-1 to utf8 (#2552)
This fixes a bug with json_encode() being unable to produce output
because it expects utf8 strings.
2022-03-29 22:44:43 +02:00
dag
cd174c7e22 [DanbooruBridge] refactor: remove unnecessary fork of simplehtmldom (#2550) 2022-03-29 22:44:20 +02:00
quickwick
907d09f116 [GelbooruBridge] + inheriting Bridges. Switch to using Gelbooru API (#2472) 2022-03-29 22:42:09 +02:00
DRogueRonin
c6675ddeee [GroupBundNaturschutzBridge] Add bridge and adjust XPathAbstract (#2445) 2022-03-29 22:40:31 +02:00
Yaman Qalieh
98a0c2de55 [EtsyBridge] Repair bridge and flip checkbox (#2457) 2022-03-29 22:23:14 +02:00
KN4CK3R
a746987d7a [Webfail] Extract timestamp from element (#1852)
Works only for German language.
2022-03-29 20:46:55 +02:00
Michael Bemmerl
6d4155f995 [GithubTrendingBridge] Fix bridge: not all languages worked (#1615)
* [GithubTrendingBridge] Fix bridge: not all languages worked

Languages with more than one word (like "Common Lisp") were not working. Looks like GitHub changed the parameter format: white space is encoded with dashes.

This prompted me to update all languages while I was at it. This also fixed the bug that the C# & F# languages were not working, because the # has to be URL encoded, which is now done in the parameter value. The language "Ren'Py" was commented out. Probably because the single quote was not escaped? I also fixed that.

* [GithubTrendingBridge] Fix PHP notice.

A repo owner can leave the repo description empty, which means the HTML element isn't there. In this case the code produced a PHP notice. This is fixed by checking for null.

* Changed getName() to retrieve the language name directly from the PARAMETERS.

Co-authored-by: dag <me@dvikan.no>
2022-03-29 20:15:18 +02:00
Bocki
58f9e41e0b [core] Change comment behavior (#2558) 2022-03-28 23:04:38 +02:00
csisoap
e86ce338a2 [ReutersBridge] Updated 'Top News' feed, some fix (#2488) 2022-03-28 20:34:41 +02:00
Mickaël Schoentgen
626cc9119a Update CryptomeBridge.php (#2555) 2022-03-28 17:18:17 +02:00
Bocki
44af64d3aa [Docker] Debug addition fixed (#2551) 2022-03-28 01:30:24 +02:00
dag
90db8c4969 [WordpressBridge] fix: add css selector for article, #2173 (#2545)
* [Wordpress] fix: add css selector for article, #2173

* fix: resolve relative links in item content
2022-03-28 00:20:44 +02:00
Bocki
8e423277e0 [core] Update pr html generator (#2549) 2022-03-27 23:35:13 +02:00
Glandos
fe43537b45 [PhoronixBridge] support multipage and embed benchmarks (#2522) 2022-03-27 13:45:32 +02:00
Thibault Couraud
87533222c7 [FindACrewBridge] Fix bridge (#2541) 2022-03-26 19:10:48 +01:00
Stelfux
91b8e4196e [FeedExpander.php] Preserve original icon (#2145) 2022-03-26 19:09:27 +01:00
Dag
74ec1b5687 [LWNprevBridge] fix: broken bridge 2022-03-26 03:17:46 +01:00
Dag
94e6feced2 Merge branch 'master' of github.com:RSS-Bridge/rss-bridge 2022-03-26 02:31:23 +01:00
Dag
b144ab2bd7 [HeiseBrige] fix: broken bridge
This is a feed expander and heise sometimes includes
feed items which point to https://www.techstage.de
for which we dont have parsing for.
2022-03-26 02:30:21 +01:00
Yaman Qalieh
012ecf8e52 [GoogleGroupsBridge] Add new bridge for Google Groups (#2451) 2022-03-26 01:09:33 +01:00
Dag
4d4ce3f380 [Arte7Bridge] test: use legal default value for checkbox 2022-03-26 00:07:34 +01:00
Dag
2c00ecb923 [Arte7Bridge] feat: add duration filter #662
The feed item was given a "duration" key but that's not used
for anything.

refs https://github.com/RSS-Bridge/rss-bridge/issues/662
2022-03-26 00:03:38 +01:00
quickwick
02ba3adcc9 [EZTVBridge] Switch to using EZTV API (#2476) 2022-03-25 22:21:47 +01:00
quickwick
37e3d6f2f6 [CBCEditorsBlogBridge] New bridge (#2487) 2022-03-25 21:35:06 +01:00
Joseph
7f4a0fae0c [YouTubeCommunityTabBridge] Add Bridge (#1594) 2022-03-25 21:31:39 +01:00
Dag
33da1476c9 [AcrimedBridge] feat: add limit option
This change preserves the prior behavior of
fetching all items.

This particular feed always has exactly 10 items.
2022-03-25 20:59:23 +01:00
Eugene Molotov
364cc8d0b8 [docs] InstagramBridge: adapt bridge documentation to new documentation structure (#2538) 2022-03-26 00:07:18 +05:00
Bocki
4c947211d2 [core] prtester debug mode (#2537) 2022-03-25 19:56:17 +01:00
Bocki
c46ff51c51 [core] Adapt pr tester 2022-03-25 19:38:26 +01:00
Bocki
608723f95c [core] Adapt pr tester (#2536) 2022-03-25 19:31:11 +01:00
Eugene Molotov
25081eedba [InstagramBridge] Documentation for configuring this bridge (#2437) 2022-03-25 23:25:35 +05:00
Bocki
aff442de1b [core] Add pr-html-generator (#2525) 2022-03-25 16:56:38 +01:00
Yaman Qalieh
105fbe9dda [PlantUMLReleasesBridge] Bridge optimizations (#2459) 2022-03-25 16:44:42 +01:00
Dag
3187592dba fix: add a few example/default values 2022-03-25 15:33:34 +01:00
Dag
2bd3f22dd5 [IPBBridge] fix: bug in feed detection logic
The previous author forgot to also append ".xml"
when actually fetching the detected feed.
2022-03-25 14:49:57 +01:00
Eugene Molotov
35b905c074 [core] Re-enable phpunit tests (#2393) 2022-03-25 12:09:05 +01:00
Yaman Qalieh
197149d90b [CraigslistBridge] Add new bridge (#2479) 2022-03-25 10:58:54 +01:00
Bocki
f11e792f84 [maintenance] Fix tests (#2532) 2022-03-25 10:41:27 +01:00
Jacob Zelek
071412130b [SummitsOnTheAir] New bridge (#2096) 2022-03-25 09:48:55 +01:00
Dag
8b59772be3 [ElsevierBridge] fix: typo in exampleValue 2022-03-25 03:42:05 +01:00
Dag
6e0589f9a0 [EconomistBridge] fix: broken bridge
Fixes: Call to a member function find() on boo

The new-style articles had their DOM changed.
2022-03-25 03:07:35 +01:00
Dag
b57d19b29c [DesoutterBridge] fix: default value 2022-03-25 02:29:51 +01:00
Dag
dbd480e2c0 [AsahiShimbunAJWBridge] fix: use case sensitive list value
This bug happened not in web UI but when extracting the default
value in order to do automatic testing.
2022-03-25 02:04:55 +01:00
Dag
35afee6103 [ABCNews] fix: broken css selector 2022-03-25 01:41:40 +01:00
Yaman Qalieh
32a6348418 [NordbayernBridge] Fix linting issue (#2531) 2022-03-25 01:26:57 +01:00
quickwick
b5ab2ee676 Add stickers endpoint for search (#2483) 2022-03-25 01:25:53 +01:00
Yaman Qalieh
acef0ab5cc [WallpaperStopBridge] Delete bridge (#2458)
This website is no longer serving content
2022-03-25 01:05:05 +01:00
Yaman Qalieh
e0d99f2a84 [DavesTrailerPageBridge] Add timestamps to feed (#2456) 2022-03-25 01:02:16 +01:00
Niehztog
55acf661b9 add support for more media types as enclosures, handle result of /tex… (#2324) 2022-03-25 00:30:14 +01:00
eggwhalefrog
3a9e528301 [NordbayernBridge] add author & timestamp of article (#2309) 2022-03-25 00:28:06 +01:00
Florent Machen
297a6cf191 [WorldCosplayBridge] fix Cosplayer API response structure (#2307) 2022-03-25 00:27:23 +01:00
Florent Machen
9cd8e93bb9 [GQMagazineBridge] fix retrieve the content of an article at a given url (#2305) 2022-03-25 00:26:38 +01:00
Dag
943a5e3e8b [CryptomeBridge] Fix pageformat (#2239) 2022-03-25 00:11:28 +01:00
Yaman Qalieh
2ade568a84 [WikipediaBridge] Add Russian Version (#2529) 2022-03-25 00:02:38 +01:00
Dag
50bab079e1 feat: add new bridge HashnodeBridge (#2231)
https://github.com/RSS-Bridge/rss-bridge/pull/2231
2022-03-24 23:58:17 +01:00
Yaman Qalieh
bb06826680 [FolhaDeSaoPauloBridge] Fix Linting Issue (#2528) 2022-03-24 23:36:23 +01:00
Dag
534864f47b Revert "WikipediaBridge: Added russian version (#2184)"
Was buggy.

This reverts commit f7af2beb79.
2022-03-24 23:34:43 +01:00
NikNikYkt
f7af2beb79 WikipediaBridge: Added russian version (#2184) 2022-03-24 23:32:33 +01:00
Yaman Qalieh
76ade41543 [No Squash] Fix Linting (#2527) 2022-03-24 23:24:55 +01:00
somini
cb4bc57c72 [FolhaDeSaoPauloBridge]: Small improvements (#1724) 2022-03-24 23:16:02 +01:00
sysadminstory
5c69577253 [ZoneTelechargementBridge] Fix links (#2526) 2022-03-24 23:13:40 +01:00
dawidsowa
78a5136cc9 bridges: change 'tags' to 'categories' (#1942) 2022-03-24 23:04:12 +01:00
Michael Bemmerl
1f2b295bf3 [BundestagParteispendenBridge] Add bridge to get the latest donations (#1613) 2022-03-24 22:42:15 +01:00
Michael Bemmerl
e89b4287b8 [SchweinfurtBuergerinformationenBridge] Add new bridge (#1610) 2022-03-24 22:37:44 +01:00
µKöff
02ab11121b [LaTeX3ProjectNewslettersBridge] New Bridge (#1589) 2022-03-24 22:26:19 +01:00
Paroleen
3d570761e5 [SpotifyBridge] Add new bridge (#1535) 2022-03-24 21:58:53 +01:00
Joseph
1ae7cf6530 [ScribdBridge] Fix bridge (#1478) 2022-03-24 21:32:16 +01:00
Jakub Valenta
8e2b65556f [Config] Don't check PATH_CACHE for memcached (#1489) 2022-03-24 21:29:16 +01:00
Joseph
0d20e9a05c [BandcampDailyBridge] Add Bridge (#1485) 2022-03-24 21:21:57 +01:00
Antoine Turmel
6a72432f76 Proposition : Open new feeds in a new tab #1389 2022-03-24 20:37:34 +01:00
Joseph
296ff9c63a [KilledbyGoogleBridge] Add bridge (#1373) 2022-03-24 20:03:18 +01:00
Bocki
2bba89d0f5 [DonnonsBridge] Fix linting error (#2524) 2022-03-24 12:15:22 +01:00
Bocki
b1c36da14e [GiphyBridge] Add examplevalue (#2523) 2022-03-24 12:11:01 +01:00
Bocki
1a8d0babd1 [Multiple] Fix all exampleValues and required variables (#2296) 2022-03-24 11:59:34 +01:00
Dag
f766193106 [IndeedBridge] fix: broken bridge
The html was reworked.
2022-03-24 13:52:02 +05:00
Dag
b6d1c7a58f fix: php notice
Fixes:
Notice: Undefined variable: message in /home/rssbridge/public/lib/contents.php on line 39
2022-03-24 13:52:02 +05:00
Dag
f34e09e93b [GithubGist] fix: broken css selector for title 2022-03-24 13:52:02 +05:00
Dag
384790537b fix: bug in cloudflare response detection
The cloudflare server header was not recognized in
some cases such as when the server header is "server"
or when the header value is "Cloudflare".
2022-03-24 13:52:02 +05:00
Dag
7bdc53125c fix: properly verify the existence of the curl module
This fixes a bug where it didnt use curl from cli
even though it's installed.

I believe this preserves the original intention to
not require the curl module to be installed.

https://github.com/RSS-Bridge/rss-bridge/pull/979
2022-03-24 13:52:02 +05:00
Binnette
076c413d3e Fix DonnonsBridge image not showing (#2521) 2022-03-24 13:51:55 +05:00
mw80
26f0380aaa [InstagramBridge] Add detectParameters (#1476) 2022-03-23 08:09:59 +05:00
Dag
14a7516625 feat: add new bridge StandfordSIRbookreviewBridge (#1638) 2022-03-23 00:43:26 +01:00
Dag
c30c0200d5 [TheTVDBBridge] fix: remove dead bridge
https://github.com/RSS-Bridge/rss-bridge/pull/1482
2022-03-23 00:07:41 +01:00
Dag
e01d9d1700 Squashed commit of the following:
commit 81d7934ab9
Author: µKöff <muekoeff@muekev.de>
Date:   Thu May 28 14:55:00 2020 +0200

    [FunkBridge] New Bridge
2022-03-22 22:42:15 +01:00
Joseph
d41aa84b13 [BridgeCard] Use full bridge name in data-ref tag (#1560)
Updates the data-ref tag of each bridge card to use the bridge's full name (eg. Apple Music) instead of its filename (eg. AppleMusic). This fixes issues with the search not returned some bridges.
2022-03-22 22:08:26 +01:00
Dag
6211a2cd37 [TagBoardBridge] fix: remove dead bridge
https://github.com/RSS-Bridge/rss-bridge/pull/1474
2022-03-22 21:38:16 +01:00
Bocki
76f5de3d0f [Documentation] Move all wiki pages into the repo and make it pretty (#2494) 2022-03-22 21:33:29 +01:00
t0stiman
16470e8119 [CarThrottleBridge] add bridge for carthrottle.com (#2514) 2022-03-22 21:15:40 +01:00
Dag
1fd3b12512 [ExecuteProgramBridge] style: remove execution bits 2022-03-22 21:11:18 +01:00
Dag
5aa163e7d6 [GettrBridge] fix: don't use php7.2 feature
The constant "JSON_THROW_ON_ERROR" is not present in
PHP version 7.2 or earlier
2022-03-22 21:08:15 +01:00
Bocki
ec90bd905e Add debug case (#2292) 2022-03-22 20:47:40 +01:00
dag
b646afffff [ExecuteProgramBridge] Add new bridge for www.executeprogram.com (#2339) 2022-03-22 20:46:59 +01:00
dag
05c31f49ce [ETTVBridge] fix: remove bridge ETTVBridge (#2511)
They went dead in Feb 2022.

Piracy Icon ETTV Officially Shuts Down Due to a Lack of Funds.

https://torrentfreak.com/piracy-icon-ettv-officially-shuts-down-due-to-a-lack-of-funds-220206/
2022-03-22 20:43:55 +01:00
dag
0b123ef8be [GettrBridge] Add new bridge for gettr.com (#2495) (#2505) 2022-03-22 20:43:31 +01:00
dag
cd5c59b84c [ARDMediathekBridge] remove timezone modification (#2507) 2022-03-22 20:42:54 +01:00
dag
e8db2479b5 [GithubTrendingBridge] fix: the description selector was broken (#2513) 2022-03-22 20:41:59 +01:00
dag
c87f4631f2 [core] feat: improve date rendering in html formatter (#2516) 2022-03-22 20:41:13 +01:00
Bockiii
ac8e94ec56 [EconomistBridge] Fix for new layout (#2489) 2022-03-23 00:24:07 +05:00
dag
1a3419a2d4 [GiphyBridge] Lazy load images (#2512)
This change instructs browsers to gradually load images
as the user is scrolling down. This is good for performance
because browsers wont download all images right away.
2022-03-21 00:43:25 +05:00
sysadminstory
ad6549efec [ZoneTelechargementBridge] (#2503)
The website keeps activating the Cloudflare protection even on the "non"
standard URL.

To bypass the Cloudflare protection, the bridge tries to resolve a bucnh
of known subdomains, and check if the IP is in a Cloudflare IP range.

If one of the subdomains has a "non Cloudflare" IP, then we use the
CURL_RESOLVE Curl option to connect the unprotected URL using the real
IP instead of the Cloudflare IP.

I hope this will help the bridge to work without needing a fix every week !
2022-03-18 14:01:46 +05:00
dag
3638b5553a phpcs: allow short array syntax (#2506)
Short array syntax was added to PHP in PHP 5.4 (2012) and replaces array() with []
2022-03-18 13:58:47 +05:00
User123698745
a7e70926f9 [Docker file] Fix wrong version string in docker images (#2497)
do not fully ignore git directory when building docker images
".git/HEAD" and ".git/refs/heads/*" are required by "getVersion()" in "lib/Configuration.php" to build the version string
2022-03-14 05:48:40 +05:00
Paul Staroch
18504f2356 [lib/contents.php] Use variable name 'retVal' instead of 'retval' as variable names are case-sensitive (#2498) 2022-03-14 05:46:30 +05:00
Eugene Molotov
05273a9278 [core] Make getContents exceptions to be handled correctly and correct exception message (#2447)
This commit fixes following issues:
1. 'Unexpected response' error message was returned, even if upstream did not return anything
2. Inability to handle non-20x messages with checking response body
2022-03-12 01:18:01 +05:00
Tomasz Kane
2e88955648 [CdactionBridge] Add missing channels (#2477) 2022-03-10 02:03:21 +05:00
Mynacol
cbef3b3360 [HeiseBridge] Properly extract authors (#2466) 2022-03-05 23:51:03 +05:00
Jonathan Kay
9564e9291f [ComicsKingdomBridge] Grab the last meta og:url content instead of first (#2484)
There are now two og:url values on the page, the first no longer contains the necessary date URL for the bridge to work.  Finding the last og:url on the page restores the bridge to working order.
2022-03-05 23:47:04 +05:00
sysadminstory
ad1ef3425a [ZoneTelechargementBridge] Fix protected links URL (#2481)
Links URL have been changed: the rewriteProtectedLink function is now updated !
2022-03-03 10:57:12 +05:00
csisoap
3bd4b0d6ab [ReutersBridge] Fix unexpected behaviour with article (#2478)
Sometimes, there are some articles that redirected to another site,
cause the bridge to fail.

Also about disable UID, Reuters frequently updated their article and my
feed reader don't update, I think it's maybe its UID.
2022-03-02 09:50:02 +05:00
Yaman Qalieh
6585ebc89b [ActionFactory] Prevent leaking working directory (#2480) 2022-03-01 14:14:53 +05:00
Corentin Garcia
d94bb08259 [RainbowSixSiegeBridge] Fix bridge (#2475) 2022-02-27 23:33:46 +05:00
Mynacol
2811bdc054 [HeiseBridge] Consistently use seite=all parameter (#2465)
This also filters out the parameter wt_mc=rss.red.ho.ho.atom.beitrag.beitrag from the item uri.
2022-02-24 23:41:42 +05:00
Joseph
0cf9da927e [CodebergBridge] Fix bridge (#2464) 2022-02-24 00:31:08 +05:00
Eugene Molotov
73a5dd928a [TwitterBridge] Don't decode HTML entities for feed content (#2470) 2022-02-24 00:28:29 +05:00
Yaman Qalieh
680fa29668 [ContainerLinuxReleasesBridge] Delete bridge (#2455)
CoreOS has been discontinued and coreos.com has been taken down (it redirects to redhat.com)
2022-02-19 17:43:04 +05:00
Loïc Fürhoff
765af484bc [RtsBridge] Add new bridge for Radio Télévision Suisse (#2442) 2022-02-17 08:15:19 +05:00
Jonathan Kay
7252252e3c [ComicsKingdomBridge] Fixes to accomodate new layout and site changes (#2444) 2022-02-13 12:27:41 +05:00
sysadminstory
3c18784576 [ZoneTelechargementBridge] Follow site changes (#2426) 2022-02-12 12:59:54 +05:00
Bockiii
3cde07db10 [Dockerfile] Rebase on php:7-apache-buster (#2446) 2022-02-12 09:27:01 +05:00
Yaman Qalieh
8723647513 [GooglePlayStoreBridge] Add bridge (#2110) 2022-02-12 09:17:12 +05:00
Eugene Molotov
f54c996e0f [CI] Add check, if php files are marked as non-executable (#2439) 2022-01-30 14:20:47 +05:00
Tomasz Kane
09fac3aa35 [CdactionBridge] Add new bridge (#2431) 2022-01-30 13:52:00 +05:00
ORelio
c1c998dd13 [GBAtempBridge] Fix content extraction (#2314)
Bridge was broken since GBAtemp's Xenforo 2 upgrade on 2021-09-23
2022-01-29 10:29:01 +05:00
Eugene Molotov
fb19142a54 [InstagramBridge] Add options to reduce 429 errors
First option is session_id of existing Instagram account.
Second option is customizing cache timeout for InstagramBridge.

Those options can be combined.
2022-01-26 00:35:15 +05:00
Eugene Molotov
9be00ff84e [core] Load bridge configuration immediately after creating bridge object
Primary reason is allowing to load configuration
params, when executing getCacheTimeout
2022-01-26 00:32:37 +05:00
Mitsu
918041cc28 [FDroid] minor syntax fix for phpcs 2022-01-24 12:41:33 +01:00
Mitsu
e9f871ce68 [FDroid] cache up, add timestamp extraction
- increase caching from 2 to 4 hours
- using cURL, extract Last-Modified header of app icons and use as item timestamp
Test warning: F-Droid response time is quite slow even on static assets, the additional requests might impact bridge performance further
2022-01-24 12:36:49 +01:00
Sandro
018fd1c8f2 [GithubPullRequestBridge] Sort by newest PRs instead of latest updated (#2064) 2022-01-24 10:58:54 +05:00
Eugene Molotov
30553d8665 contrib: lint fetch_contributors.php 2022-01-20 10:22:10 +05:00
Eugene Molotov
5df8bf956a Bump version to dev.2022-01-20 2022-01-20 10:17:59 +05:00
Eugene Molotov
a49767e71b [README] Update list of contributors 2022-01-20 10:17:06 +05:00
Eugene Molotov
0584fdddde README: add buttons to irc and matrix 2022-01-20 10:12:08 +05:00
Eugene Molotov
0fdd281fb2 README: take out debian and guix buttons, since they are not officially maintained 2022-01-20 09:45:53 +05:00
Eugene Molotov
0e17282f60 contrib: add somehow working emacs script to help prepare new release 2022-01-20 09:43:01 +05:00
Eugene Molotov
0f1ec4a879 contrib: save template for creating new releases 2022-01-20 09:43:01 +05:00
Eugene Molotov
e9d3d7ba67 [contrib] Place script for fetching contributors from GitHub's gist to contrib directory
No need to depend on gist.github.com
2022-01-20 09:43:01 +05:00
岳东辰
3cef35a432 [ABCNewsBridge] New bridge (#2255) 2022-01-17 10:35:01 +05:00
Eugene Molotov
2f10d2345a [TwitterBridge] Meet the new maintainer: arnd-s 2022-01-16 23:42:28 +05:00
arnd-s
02a8ae4c62 [TwitterBridge] Use Twitter API V1.1 guest/activate for requesting new guest tokens (#2414)
Instead of searching inside base html page for the guest token, this patch instead uses the Twitter API V1.1 to acquire guest tokens
2022-01-16 07:59:02 +05:00
Štěpán Škorpil
084a1bcf19 [CeskaTelevizeBridge] Follow website changes (#2420) 2022-01-16 07:54:20 +05:00
Dag
418f951dd1 [ThePirateBayBridge] Repair broken bridge
This is more like a refactor because they dont serve data in plain
html anymore. Instead, the data is available from a json api
at apibay.org

Could possibly expand this bridge because their api has more to give.

I learned about this api by grokking https://thepiratebay.org/static/main.js
and by looking at browser ajax requests.

For some unknown reason they host some static assets at
https://torrindex.net/ which is used by the bridge to render
magnet image and user status image.

Signed-off-by: Dag <me@dvikan.no>
2022-01-15 11:18:57 +05:00
Bockiii
9dcce0ba1d [EconomistBridge] Fix if no article image present (#2328) 2022-01-15 10:29:18 +05:00
Joseph
607d9297ff [DuckDuckGoBridge] Fix bridge (#2335) 2022-01-15 10:24:13 +05:00
Joseph
c65feffb61 [FirefoxAddonsBridge] Fix add-on download links (#2338) 2022-01-15 10:16:43 +05:00
MarKoeh
f259fa7f9f [ARDMediathekBridge] Switch to JSON-API (#2380)
* Switch ARDMediathekBridge to JSON-API

The html screen scraping approach of ARDMediathekBridge did not work reliably. I could not find one show for which the item list was not empty using the html screen scraping approach.

The proposed change uses the JSON-API of the WebApp. Although there is still room for improvement (feed title, better understanding of the API, more accurate mimic of the webapp's behavior, de-pagination …), it does work with this change.

Indicate that now full URLs as well as just the ID are accepted.
2022-01-10 11:47:49 +01:00
sysadminstory
368a198321 [PepperBridge] fix and discussion (#2383)
* [PepperBridge] Fix some Notice for expired items

Some expired iteams are not identical to normal items for the title and
URI, so they got a special handling to remove PHP Notice.

The "most commented" sorting option was removed and show now some forum.
So it was removed.

With DealabsBridge, MydealsBridge and HotUKDealsBridge, you can monitor
a discussion for new comments, excluding or not post without URL
2022-01-10 11:46:26 +01:00
Albirew
590fdd9f9b [HentaiHavenBridge] domain and content retrieval update (#2402) 2022-01-10 11:44:41 +01:00
jNullj
799c93a3c6 [ExplosmBridge] Rewrite to work without feedburner (#2417)
* [ExplosmBridge] Rewrite to work without feedburner

re-wrote the bridge to scrap from the new explosm site as the old method of using feedburner is not working anymore, feedburner is stuck on dec/22 when the explosm site changed.
2022-01-10 11:44:18 +01:00
Quentin de Longraye
2f957b6870 [CI] Tag docker images with the commit sha (#2418)
We do not release stable tags often. To avoid using `latest`, for instance in Kubernetes manifests, this change allows to reference a repository commit. This way it is easy to lock image to a specific commit, and try and rollback if a change to a newer commit brings regressions.
2022-01-09 14:09:28 +05:00
arnd-s
12ff697ab0 [TwitterBridge] Continue using existing guesttoken from cache, when requesting a new one failed (#2396) 2022-01-06 10:59:37 +05:00
somini
8530aa54f2 [ComboiosDePortugalBridge] Temporarily ignore certificate checks (#2403) 2022-01-05 04:44:59 +05:00
Eugene Molotov
d0ef8aa71d [CI] Add support for php 8.0 and 8.1 (#2405) 2022-01-05 04:40:30 +05:00
Eugene Molotov
59e77a9e51 [README] Clarify problems with InstagramBridge and FacebookBridge (#2401) 2022-01-03 11:15:45 +05:00
Eugene Molotov
37cb4091d4 bridges: remove redundant "or returnServerError" after getContents/getSimpleHTMLDom/getSimpleHTMLDomCached (#2398)
When fetching website contents, exceptions already raise on fetching error
2022-01-02 14:36:09 +05:00
Christian Schabesberger
fc51c6753d [NordbayernBridge] Fix banner images (#2384) 2022-01-02 03:51:59 +05:00
sysadminstory
71cd15c35d [AutoJMBridge] Rework of the script to handle the new website (#2390)
The brand - model page does not have filters now, so the actual feed
using this bridge are broken.

Website has changed. The new website offers a new search function with
many more filter. So I switched to this new search page for this bridge.
2022-01-02 02:15:13 +05:00
Eugene Molotov
df408fb8bc [CI] Temporarly disable phpunit7 until it starts working again (#2392) 2021-12-26 09:25:36 +05:00
Shikiryu
e545f43a67 [KhinsiderBridge] Add bridge (#2302) 2021-12-20 12:36:44 +05:00
dag
ec55e99934 [HackerNewsUserThreadsBridge] Repair broken bridge (#2344) 2021-12-19 14:18:57 +05:00
dag
67e33186ce [GiphyBridge] Repair broken bridge (#2347) 2021-12-18 15:26:51 +05:00
sysadminstory
a0bbbd6978 [RadioMelodieBridge] Fix to use the new website layout (#2330) 2021-12-18 15:19:58 +05:00
Joseph
2ee091665a [ASRockNewsBridge] Fix bridge (#2373) 2021-12-18 15:17:16 +05:00
triatic
35930ee4e4 [TwitterBridge] Increase guest token expiry time (#2374) 2021-12-18 14:54:18 +05:00
Aaron F
e1290aa42c [CVEDetailsBridge] Add bridge (#2332)
CVE Details is a collection of CVEs, taken from the National Vulnerability
Database (NVD) and other sources like the Exploit DB and Metasploit. The
website categorizes it by vendor and product and attach the CWE category.
There is an Atom feed available, but only logged in users can use it,
it is not reliable and contain no useful information. This bridge create a
sane feed with additional information like tags and a link to the CWE
a description of the vulnerability.
2021-12-18 09:44:05 +05:00
dag
814711e3af [ScmbBridge] Remove "read more" text only if it exists (#2368) 2021-12-16 10:06:52 +05:00
Eugene Molotov
28db707587 [OpenClassroomsBridge] sebsauvage does not maintain this bridge
Reference: https://github.com/RSS-Bridge/rss-bridge/issues/2333#issuecomment-989611085
2021-12-09 15:26:31 +05:00
dag
b48739d0ba [DerpibooruBridge] Fix parsing of title (#2346)
The previous value was an int and was not accepted
by rss-bridge as a title. This change uses the image
name instead which fixes the problem and is a better
title than the image id.
2021-12-08 17:17:28 +01:00
csisoap
b9d92150e1 [ReutersBridge] Migrate to new API (#2348)
* [ReutersBridge] Migrate to new API

- Add new API, feeds.
- Old feed name are perserved for backward compatibility.
- Remove 'Special Report' feed.
- Some feed continue to use old Wire API due to not available in new one.
- Add some new type of content, replace iframe with blockquote for twitter.
2021-12-08 17:16:40 +01:00
Bockiii
b395fe2641 [core] Implemented feature to read config from environment variables (#2100) 2021-12-03 11:15:08 +05:00
Eugene Molotov
4bc534c80f [FacebookBridge] teromene and logmanoriginal do not maintain this bridge defacto 2021-12-03 05:31:46 +05:00
Eugene Molotov
071fdef599 [core] Drop php 5.6 and php 7.0 support (#2224) 2021-12-03 04:12:16 +05:00
erik
ae6a3227b0 [FlashbackBridge] Add new bridge (#2343) 2021-12-03 03:57:21 +05:00
benzel
f469489b56 [AppleAppStoreBridge] Add Germany (#2350) 2021-12-03 03:56:06 +05:00
dag
490f556783 [TheCodingLoveBridge] Remove redundant bridge (#2342)
This is now a wordpress blog and their official feed is at https://thecodinglove.com/feed
2021-11-29 11:26:13 +05:00
somini
192f0278d2 [PanacheDigitalGamesBridge] Add bridge (#2321) 2021-11-06 23:58:30 +05:00
sysadminstory
2b634002f2 [DealabsBridge - HotUKDealsBridge - MydealsBridge] Set the Feed URL according to the parameters (#2320) 2021-11-06 23:43:44 +05:00
Christian Schabesberger
42379071e9 [NordbayernBridge] Fix banner URL (#2326)
* make banner images show for nordbayern again
* make author portrait not apear as article banner for nordbayern
2021-11-06 23:28:12 +05:00
Joseph
fd54042ef3 [BandcampBridge] Add support for labels (#2286) 2021-10-30 01:43:20 +05:00
Martin Leyrer
3764348b76 [core] Accept additional "successful" 2xx status codes (#2310) 2021-10-30 01:38:25 +05:00
dotter-ak
0ba0e2de4e [UrlebirdBridge] Fix for non ASCII characters in post URI (#2312) 2021-10-30 01:36:09 +05:00
Florent Machen
4187d8f4cf [GitHubGistBridge] fix use the css selector "contains" to find a class in the middle of the utility classes (#2306) 2021-10-30 01:32:31 +05:00
ORelio
1c6532a9d0 [NextgovBridge] Update categories, fix missing element (#2316) 2021-10-30 01:28:02 +05:00
ORelio
547829f971 [FuturaSciences] Improve content extraction (#2317)
- Fix tracking removal in URL
- Fix images broken due to new lazy loading mechanism
- Remove headline, articles do not have it anymore
- Improve article cleanup
2021-10-30 01:24:19 +05:00
ORelio
970bdd45f9 [DarkReadingBridge] Fix content extraction (#2315)
Also:
- Add article limit (main feed was broken due to too many articles)
- Add support for article thumbnail
2021-10-30 01:21:07 +05:00
Eugene Molotov
b86ed70376 [core] Backported str_starts_with, str_ends_with and str_contains from php 8 (#2318) 2021-10-30 01:06:04 +05:00
Eugene Molotov
9254d14f50 [VkBridge] Multiple fixes:
- Correct video title
- Do not add repost of deleted post to feed
2021-10-25 10:16:24 +05:00
Matt DeMoss
8f98e07979 [PcGamerBridge] Use meta tags to generate feed contents (#2271) 2021-10-19 11:53:26 +05:00
Bockiii
8d0fc54e4d [FSecureBlogBridge] Limit number of returned items (#2300) 2021-10-19 08:23:38 +05:00
csisoap
bdf15c3ce0 [UnogsBridge] Add fallback if not found any high-res image (#2301) 2021-10-19 08:22:17 +05:00
sal0max
927b08ed00 [LegoIdeasBridge] Add bridge (#2284)
* [LegoIdeasBridge] Add bridge
2021-10-14 22:52:42 +02:00
Niehztog
87b3aaa550 [XPathAbstract] Fix encoding of feed output (#2297) 2021-10-14 14:18:00 +05:00
Yaman Qalieh
c445ba6ebb [MozillaBugTrackerBridge] Fix incorrect newlines in feed title (#2298) 2021-10-14 14:13:18 +05:00
Bockiii
793c55f43d [Docker] Allow to define port via HTTP_PORT (#2285) 2021-10-09 22:02:38 +05:00
Bockiii
11be390e65 [MozillaSecurityBridge] Limit items to 20 (#2287) 2021-10-09 16:03:12 +05:00
somini
ba5baaf7c9 [JornalDeNoticiasBridge] Add bridge (#2293) 2021-10-07 16:15:21 +05:00
sysadminstory
f0ddd686e3 [ExtremeDownloadBridge] Update URL (#2290) 2021-10-05 12:52:29 +05:00
sysadminstory
f6b9864bdd [ZoneTelechargementBridge] Update website URL (#2289) 2021-10-05 12:51:26 +05:00
KamaleiZestri
64b7c54bc8 [BakaUpdatesMangaReleasesBridge] Added option to display releases based on a user list (#2276)
* [BakaUpdatesMangaReleasesBridge] Added option to display releases based on a user list
2021-10-03 22:35:31 +02:00
KamaleiZestri
d5a010adcd [PillowfortBridge] Added new bridge (#2275)
* [PillowfortBridge] Added new bridge
2021-10-03 21:59:33 +02:00
Bockiii
2a609b39bd [LinkedInBridge] Remove bridge (#2269) 2021-10-03 21:00:24 +05:00
Bockiii
dacc586dca New docker build mechanism (#2268)
* New docker build mechanism
2021-10-03 16:06:30 +02:00
csisoap
8bcf4ebfbf [NationalGeographicBridge] Rewrite bridge (#2177)
- All the option will be preserved.
- Add timestamp, author's name included with full article.
2021-10-01 18:39:36 +05:00
csisoap
cb111a3ebd [UnogsBridge] Better feed title for Country context (#2279) 2021-09-28 14:20:04 +05:00
Timendum
42e40e2823 [PicukiBridge] Fix image URLs in content (#2282)
URLs in Picuki are already absolute
2021-09-28 14:15:41 +05:00
Bockiii
1ddce120ae [EconomistBridge] Full rework (#2272) 2021-09-26 16:25:19 +05:00
Joseph
927cb17dbf [SoundcloudBridge] Add support for albums, reposts & likes (#2236) 2021-09-20 17:53:41 +05:00
D5k H3h
ccb2e64fd0 [WallpaperflareBridge] Add bridge (#2179) 2021-09-11 13:27:17 +05:00
csisoap
a26408594b [YoutubeBridge] New features, fixes and refactors (#2208)
New features:
- Add support for custom channel name.

Fixes:
- In playlist mode last uploaded videos were not in feed
- Search mode returned empty feed

And a lot of refactoring
2021-09-11 13:20:14 +05:00
Bockiii
324932642d [PokemonTVBridge] Add bridge (#2219) 2021-09-05 12:11:36 +05:00
Bockiii
455b5e09a1 [Dockerfile] Add custom config location (#2098) 2021-09-05 08:10:54 +05:00
Bockiii
bcc15228d8 [RedditBridge] Return back NSFW posts to feed (#2257)
As it was working before applying https://github.com/RSS-Bridge/rss-bridge/pull/2229
2021-09-04 18:00:02 +05:00
Eugene Molotov
68d9e2ff24 [NineGagBridge] Remove whitespace on wtf section
Don't know why github actions did not run on https://github.com/RSS-Bridge/rss-bridge/pull/2094
Certainly wtf
2021-08-25 18:16:00 +05:00
Bockiii
a5d33615f5 [RedditBridge] Add keyword search function (#2229) 2021-08-25 18:09:36 +05:00
Christian Schabesberger
8f634eb4a1 [NordBayernBridge] Fix election articles (#2253) 2021-08-25 18:08:23 +05:00
Bockiii
677e4974d1 [NineGagBridge] Updated sections and added video 2021-08-25 18:06:10 +05:00
Joseph
10c5259493 [OpenlyBridge] Add bridge (#2129) 2021-08-15 23:42:28 +05:00
Joseph
cb3c055df9 [InternetArchiveBridge] Add detectParameters (#2142) 2021-08-15 23:36:38 +05:00
Christian Schabesberger
036a3ad245 [NordbayernBridge] Add city of Erlangen (#2248) 2021-08-15 18:57:40 +05:00
Joseph
69ce8106ce [BingSearchBridge] Remove bridge (#2242)
Microsoft has removed the bing search discover functionality.

https://www.bing.com/discover/ pages now redirect to https://www.bing.com/images/trending
2021-08-13 08:54:37 +05:00
ORelio
8a30480a45 [Releases3DSBridge] Remove requests to IGN (#2246)
This part of the bridge was meant to find game info on IGN but rarely found useful results, and is harder to maintain than the rest of this bridge due to changes at IGN.
2021-08-13 08:52:57 +05:00
Joseph
f36832b66e [CodebergBridge] Add bridge (#1951) 2021-08-13 08:51:50 +05:00
t0stiman
f3f934ed8b [HardwareInfoBridge] Add bridge (#2232) 2021-08-10 23:00:32 +05:00
Joseph
7c46c64242 [FierPandaBridge] Remove bridge (#2238) 2021-08-10 22:55:24 +05:00
ORelio
bf1773ed8b [GBAtemp] Fix news extraction (#2241) 2021-08-10 22:49:10 +05:00
Joseph
4529e3699a [BridgeImplementationTest] Allow multiple contexts to have an empty parameters array (#1954) 2021-08-10 22:37:41 +05:00
sysadminstory
f5f0b77805 [RadioMelodieBridge] Replace JS Audio Player (#2233)
The Javascript Audio Player is replaced by the plain <audio> HTML tag
2021-08-04 09:04:45 +05:00
t0stiman
3637777070 [style] add dark mode (#2029) 2021-08-01 16:31:58 +05:00
Tobias Alexander Franke
c673917aca [BinanceBridge] Fix blog posts (#2226) 2021-07-29 21:55:36 +05:00
Eugene Molotov
877707f7b2 [contrib] Add directory for unorganized bin of various useful things contributed by the community around RSS-Bridge
Idea came after reading https://drewdevault.com/2020/06/06/Add-a-contrib-directory.html
2021-07-28 00:51:34 +05:00
sysadminstory
2689f5f7fa [DealabsBridge - HotUKDealsBridge - MydealsBridge] Update groups (#2083)
The bridges has been updated with the newest "groups" available on every website
2021-07-28 00:46:01 +05:00
Bockiii
bf1cb8fadc [IndiegogoBridge] New bridge (#2135) 2021-07-28 00:41:56 +05:00
Eugene Molotov
716f5ddc0e [PikabuBridge] Do not strip strikethrough tags in body content 2021-07-24 00:39:00 +05:00
Eugene Molotov
0ee549f468 [PikabuBridge] Remove whitespace from fake news marker 2021-07-24 00:39:00 +05:00
Eugene Molotov
4a1e26fd07 [WordPressBridge] aledeg is not maintainer of this bridge 2021-07-24 00:37:57 +05:00
sal0max
e14f647075 [SpottschauBridge] New bridge (#2193) 2021-07-20 13:16:43 +05:00
Marcus
34489431b4 [PicukiBridge] New bridge. Alternative to InstagramBridge (#2183) 2021-07-20 13:06:56 +05:00
dotter-ak
126cf1a7fa [Drive2ruBridge] Fixed incorrect titles and URLs in logbooks (#2215) 2021-07-20 13:03:17 +05:00
岳东辰
3050f0ae70 [ARDMediathekBridge] New bridge (#2158) 2021-07-20 12:58:50 +05:00
csisoap
cabf7a748a [ReutersBridge] Change timestamp, add new feed, add alt text to image (#2150) 2021-07-20 12:54:07 +05:00
dotter-ak
84450e7e8d [Drive2ruBridge] Bugfix (#2211) 2021-07-13 10:20:46 +05:00
Joseph
971cd9ba39 [FirefoxAddonsBridge] Remove duplicate code (#2209) 2021-07-12 22:56:51 +05:00
Nemo
9fa782105d [AmazonPriceTrackerBridge] Fixes for subscription items (#2205) 2021-07-12 22:49:29 +05:00
dotter-ak
cb7f5b057f [Drive2ruBridge] Add news, personal blogs and featured topics (#2156) 2021-07-09 00:00:16 +05:00
csisoap
a8d1acfdad [UnogsBridge] New bridge (#2198) 2021-07-07 15:25:13 +05:00
sal0max
b5d9742a21 [NikonDownloadCenterBridge] Add bridge (#2195) 2021-07-06 18:00:19 +05:00
sal0max
7dd1a7dccc [AmazonPriceTrackerBridge] Fix bridge (#2194) 2021-07-06 01:26:08 +05:00
csisoap
1f6ad000ce [RedditBridge] Add option to choose for New, Hot and Top submissions (#2189) 2021-07-02 00:41:56 +05:00
Christian Schabesberger
398e175fe0 [NordbayernBridge] Follow site updates (#2169) 2021-07-01 07:21:58 +05:00
Corentin Garcia
0de2db853f [NYTBridge] Fix article parsing (#2106)
Co-authored-by: podiki <podiki@users.noreply.github.com>
2021-06-30 15:14:25 +05:00
sysadminstory
9399ebb2c6 [RadioMelodieBridge] Add timestamp support and content fix (#2105) 2021-06-26 01:12:33 +05:00
dotter-ak
606972dd3c [UrlebirdBridge] Add bridge (#2163) 2021-06-26 00:55:13 +05:00
Bockiii
ecaae735d9 [core] Support for bridge maintainers' donation URLs (#2102) 2021-06-26 00:45:25 +05:00
岳东辰
5598fef3cf [WikipediaBridge] Update elements (#2167) 2021-06-20 15:23:29 +05:00
triatic
9c99a1a9c1 [FacebookBridge] Increase cache timeout (#2149)
Facebook aggressively throttles queries now
2021-06-12 23:15:56 +05:00
ORelio
75cc52a62c [FilterBridge] Various improvements (#2148)
- Add option for case-insensitive regex
- Allow matching item content or author in addition to item title
- Optionally attempt to convert encoding when applying matches
2021-06-07 23:11:12 +05:00
Eugene Molotov
973e49d93e [TelegramBridge] Add test cases for detectParameters 2021-06-05 09:34:16 +05:00
Eugene Molotov
c580219627 [BridgeImplementationTest] Implement feature for testing detectParameters method 2021-06-05 09:34:16 +05:00
Joseph
a18af3952d [KATBridge] Remove bridge (#2141) 2021-06-03 00:16:12 +05:00
Joseph
15907e2bbd [StoriesIGBridge] Remove bridge (#2139)
Website has been unavailable since at least October 2020

https://web.archive.org/web/20201016100555/https://storiesig.com/
2021-06-01 10:20:11 +05:00
Joseph
459adf4790 [ExplosmBridge] Fix website URI (#2140) 2021-06-01 10:18:14 +05:00
AxorPL
d38bc18232 [Formula1Bridge] Minor fixes (#2128)
- removed useless returnServerError according to your recommendation
- made use of the caption property if present
- fixed link URL
2021-05-30 23:33:15 +05:00
Jacques Heunis
eec1163fb9 [ItchioBridge] Remove reliance on in-page timestamps (#2127)
This significantly increases the possibility of missing updates (if
files are uploaded but no file names or post contents are changed) and
of showing an update when there is none (if the post text is changed
but no new files are uploaded). However with the on-page timestamps
removed I'm not sure if there is a good way to do this more accurately
so this is good as we can do at the moment.
2021-05-30 23:12:19 +05:00
Bockiii
b074abcc0d [AppleMusicBridge] Complete rebuild for new site (#2134) 2021-05-30 23:08:39 +05:00
Joseph
2ae9793f2c [CI] Update ubuntu version (16.04 => 20.04) (#2136)
Support for Ubuntu 16.04 (Xenial Xerus) runners is deprecated and will be removed on August 1, 2021
https://github.com/shivammathur/setup-php/issues/452
2021-05-30 23:04:57 +05:00
Joseph
f02d80e141 [FirefoxAddonsBridge] Fix download link extraction (#2120) 2021-05-25 00:46:07 +05:00
Yaman Qalieh
44e01a4282 [PixivBridge] Rewrite Bridge (#2111)
Also added options:
- Search for Illustrations, Manga or Novels
- Custom Post Limit
- Choose between thumbnails and full-sized image
2021-05-25 00:42:39 +05:00
Eugene Molotov
63d257d9d0 [PikabuBridge] Cut "script" element from post body (#2125)
Also correct description and remove useless returnServerError call
2021-05-24 00:50:24 +05:00
dotter-ak
37cd071453 [Drive2ruBridge] Add bridge (#2116) 2021-05-24 00:33:46 +05:00
Christian Schabesberger
17f9c44bfc [NordbayernBridge] Fix jpeg regex (#2118) 2021-05-19 13:44:17 +05:00
Christian Schabesberger
28aaf59007 [NordbayernBridge] Exclude slideshows when handling articles (#2117) 2021-05-19 13:43:10 +05:00
Corentin Garcia
e8d241e8c9 [CourrierInternationalBridge] Switch to FeedExpander (#2107) 2021-05-17 23:36:11 +05:00
t0stiman
2b793f04de [RaceDepartmentBridge] Follow site changes (#2087) 2021-05-17 23:18:51 +05:00
Eugene Molotov
655e02e3fe [VMwareSecurityBridge] Remove bridge
There is fulltext RSS feed: https://www.vmware.com/security/advisories.xml
2021-05-17 12:01:23 +05:00
Eugene Molotov
a4bd04310f [YoutubeBridge] Several fixes and switch maintainer (#2115)
- Fix incorrectly working method for querying video info
- Partially fix playlist mode.
At least it works, if playlist has more than 15 videos. But maximum 100 video items are parsed from playlist.

Reason of switching maintainer: https://github.com/RSS-Bridge/rss-bridge/issues/2113#issuecomment-841156902
2021-05-17 01:02:45 +05:00
Bockiii
e48617530d [ExplosmBridge] Add bridge (#2092) 2021-05-13 23:35:42 +05:00
Bockiii
3585575d68 [Docker] Add support for arm32/64 (#2104) 2021-05-13 22:57:10 +05:00
somini
e79a02ac2e [PresidenciaPTBridge]: Support multiple sections (#2082) 2021-05-10 00:22:10 +05:00
Bockiii
63ebf5ceec [BridgeCard] Make HTML default, remove other buttons (#2101) 2021-05-08 15:10:16 +05:00
AxorPL
378f78d6eb [Formula1Bridge] New bridge (#2085) 2021-05-06 23:16:22 +05:00
marius851000
d7ba7782f3 [DerpibooruBridge] Make it work again (#2079) 2021-04-26 23:07:42 +05:00
Eugene Molotov
b24b5ed3ee Bump version to dev.2021-04-25 2021-04-25 15:32:35 +05:00
Eugene Molotov
f06a8ae307 [README] Update list of contributors 2021-04-25 15:30:17 +05:00
dawidsowa
4f7ef212b7 [RedditBridge] Add detectParameters (#2070) 2021-04-19 22:17:36 +05:00
dawidsowa
13e9a96cf3 [RedditBridge]: Add score filter (#2045) 2021-04-19 22:14:35 +05:00
ORelio
00a24a98be [NyaaTorrents] Rewrite as Feed Expander (#2073)
NyaaTorrents allows search criteria as URL parameters in RSS feed so we just need to expand feed items
2021-04-19 21:59:51 +05:00
FiveFilters.org
76c38332ee [TwitterBridge] Improve timeline processing for username mode (#1946) 2021-04-12 23:08:38 +05:00
Joseph
65be209a47 [TwitScoopBridge] Remove less than (<) character from item title (#2034) 2021-04-12 23:01:46 +05:00
sysadminstory
146639ffc9 [ZoneTelechargement] Update unprotected URL and Feed URL (#2065)
The unproteced URL has changed again, and has been updated.

The RSS Feed URL is now a link to the Show page and not to the Homepage anymore.
2021-04-12 23:01:09 +05:00
sysadminstory
e1c19461ca [ExtremeDownloadBridge] Feed URL updated (#2066)
The Feed URL is now a link to the TV Show and not the Homepage !
2021-04-12 22:59:16 +05:00
Harvey Christian Pacleb
ff0c7a9013 [GenshinImpactBridge] Use Asia/Shanghai time zone for article dates (#2040) 2021-04-10 13:35:34 +05:00
ORelio
b754d14698 [FeedExpander] Handle Atom enclosures (#2039) 2021-04-04 15:21:15 +05:00
sysadminstory
3423b3bbe1 [ZoneTelechargement] Change URL load method (#2044) 2021-04-04 14:31:48 +05:00
Joseph
5966cc0a9c [TheFarSideBridge] Add bridge (#1484) 2021-04-03 23:31:49 +05:00
Joseph
579bfa669c [WallmineNewsBridge] Add bridge (#2035) 2021-04-02 18:01:51 +05:00
ORelio
d61871a45e [NyaaTorrents] Allow searching by username (#2033) 2021-03-31 10:59:31 +05:00
Eugene Molotov
0c8fabeb11 [PikabuBridge] Marking posts from "Как бы Новости" section, which are funny and deliberately fake (#2032) 2021-03-30 23:06:23 +05:00
Matthieu Rakotojaona
40c84b5dc3 [HackerNewsUserThreads] New bridge (#1902) 2021-03-30 21:56:17 +05:00
Yaman Qalieh
6f75d07456 [GitHubPullRequestBridge] Add new bridge inheriting GithubIssueBridge (#2001) 2021-03-29 22:15:56 +05:00
guigot
b4f809aa44 [MondeDiploBridge] Fix blog article uri (#1961) 2021-03-25 22:15:02 +05:00
Joseph
bcecd70df7 [DownDetectorBridge] Fix bridge (#1957) 2021-03-25 22:02:45 +05:00
Joseph
a6c0874b9a [TwitScoopBridge] Fix encoding of less than character (<) (#2023) 2021-03-21 11:39:01 +05:00
somini
9e6f063cfd [PresidenciaPT]: New Bridge (#2016) 2021-03-17 21:30:47 +05:00
Joseph
f904353fd2 [InternetArchiveBridge] Fix collection links (#1551) 2021-03-16 18:07:04 +05:00
Andrea Draghetti
3aafd44079 [TelegramBridge] Display the name of the attachments (#2003)
Sometimes attachments are posted in Telegram channels without any text. 

The script recognizes a new message but does not report any text, with this commit the file names will also be included in the RSS Feed.
2021-03-16 18:04:07 +05:00
Joseph
75b85f61e7 [BandcampBridge] Fix title extraction on empty band pages (#1966) 2021-03-16 18:00:26 +05:00
Joseph
07e1e8497c [DockerHubBridge] Add detectParameters() (#1996) 2021-03-15 21:54:26 +05:00
Joseph
700813e924 [FirefoxAddonsBridge] Add detectParameters() (#1997) 2021-03-15 21:27:53 +05:00
Joseph
5c011c8d90 [TwitScoopBridge] Add bridge (#2018) 2021-03-15 21:20:02 +05:00
Joseph
8d0d08a4d8 [YeggiBridge] Fix lint error (#2019) 2021-03-15 21:15:01 +05:00
Antoine Turmel
55548dcb5f [YeggiBridge] New bridge (#1910) 2021-03-13 21:57:30 +05:00
Justin Goette
0217b270a7 [README] Fix typo (#2004) 2021-03-10 22:13:41 +05:00
Joseph
2ed34f5ebe [FirefoxAddonsBridge] Set unique id for items (#2007)
Adds unique id for each item using item title.
2021-03-09 11:16:56 +05:00
csisoap
2448ed41c9 [ReutersBridge] Add new wireitem template type (#2006)
and retain the list of parameters
2021-03-09 11:15:48 +05:00
Joseph
b25674b3a0 [HtmlFormat] Use str_ireplace() when creating feed format URLs (#2008)
Fixes feed format URLs not being created with correct format value if html feed URL uses a lowercase format value.
2021-03-08 12:17:12 +05:00
Joseph
2ce1a6365b [README] Update build status badges (#1995) 2021-02-28 18:41:03 +05:00
Joseph
30aeeb2a0c [DockerHubBridge] Add support for official images (#1999) 2021-02-28 18:26:24 +05:00
Joseph
c294a652a3 [TelegramBridge] Add detectParameters() (#1998) 2021-02-28 18:20:44 +05:00
Park0
a5f2175531 [SymfonyCasts] Added new bridge (#2000) 2021-02-28 18:17:54 +05:00
dawidsowa
569276f4ef [RedditBridge]: Add user option (#1943) 2021-02-23 12:08:43 +05:00
Joseph
687eb728d4 [DockerHubBridge] Fix bridge name (#1994) 2021-02-22 15:03:04 +00:00
Monocularity
0521ba5873 [NordbayernBridge] Fixed typo of region "Hilpoltstein" (#1962) 2021-02-21 16:43:23 +00:00
Joseph
3d642971c0 [FirefoxAddonsBridge] Add bridge (#1952) 2021-02-21 17:30:01 +05:00
Shikiryu
8f086169cc [TheYeteeBridge] Fix HTML parsing (#1986) 2021-02-21 15:19:20 +05:00
Joseph
ce34e7eb89 [DockerhubBridge] Add bridge (#1990) 2021-02-21 15:17:07 +05:00
sysadminstory
ee5d190391 [RadioMelodieBridge] Fix header image (#1985)
Header Image is now using a direct link to the image, but without the
website base URL : the bridge now sends the right URL.
2021-02-17 10:07:35 +05:00
Eugene Molotov
98352845a1 [VkBridge] Remove non ascii chars from post date to correctly parse it (#1977) 2021-02-10 18:11:48 +05:00
Joseph
9e58735b01 [FormatFactory] Ignore case in format values (#1967) 2021-02-09 18:13:03 +05:00
Thomas
771b851b52 [contents.php] Fix logical error in getSimpleHTMLDOMCached function (#1974)
Previously content was only loaded from cache when debug mode was enabled (the opposite of the expected behavior)
2021-02-09 17:40:16 +05:00
Joseph
809343ed06 [SoundcloudBridge] Fix client ID extraction (#1973) 2021-02-09 17:33:14 +05:00
Lyra
e9424f6a08 Merge branch 'master' of github.com:RSS-Bridge/rss-bridge 2021-02-07 14:44:36 +01:00
Lyra
e5846c03ba Attempt to fix LeBonCoinBridge 2021-02-07 14:44:28 +01:00
Joseph
6224fbb6a2 [BridgeCard] Display configuration options (if enabled) when bridge has no parameters (#1968)
Updates displayBridgeCard() in BridgeCard to allow configuration options noproxy and cache_timeout to be displayed, if enabled, when a bridge has no parameters in its PARAMETERS array
2021-02-05 10:17:30 +05:00
sysadminstory
eab575dc9d [ZoneTelechargement] Update Direct Download Unprotected URL (#1963) 2021-01-30 18:02:36 +00:00
André Andersson
b56637c833 [BukowskisBridge] Add bridge (#1927) 2021-01-29 23:36:45 +05:00
Simon816
005b22701d [FSecureBlogBridge] Add bridge (#1932) 2021-01-29 23:27:35 +05:00
hollowleviathan
43b7621f45 [ReutersBridge] Add bridge (#1653) 2021-01-29 22:57:40 +05:00
Joseph
ea289a0cea [GithubIssueBridge] Fix issue id and comment id extraction (#1950) 2021-01-27 11:06:59 +05:00
Joseph
43acb555e0 [ChristianDailyReporterBridge] Remove bridge (#1948)
> The time has come for the Christian Daily Reporter to ride off into the sunset. Disrn is here now and it delivers everything CDR did, times ten.

https://christiandailyreporter.com/
2021-01-22 10:20:17 +05:00
Eugene Molotov
3b7e61fb55 [Arte7Bridge] Mitsu is no longer maintainer of the bridge
> That said, please also feel free to remove me as "maintainer" since that's simply no longer reality

https://github.com/RSS-Bridge/rss-bridge/issues/1906#issuecomment-765015048
2021-01-22 09:52:07 +05:00
Corentin Garcia
fbbd6a02c6 [DribbbleBridge] Fix pictures parsing (#1911) 2021-01-20 22:26:15 +05:00
sysadminstory
3534193032 [ZoneTelechargement] Update URL and fix typos (#1936) 2021-01-17 16:32:59 +05:00
sysadminstory
81fc8c89d4 [ExtremeDownloadBridge] Update URL (#1937) 2021-01-17 16:31:44 +05:00
Eugene Molotov
ea1de07fe5 [CI] Fix non-working phpcompatibility job (#1928) 2021-01-12 12:02:49 +05:00
Eugene Molotov
2de5ce8387 [CI] Fix phpcompatibility job running wrong scenario 2021-01-12 11:00:38 +05:00
Eugene Molotov
28f9215913 [CI] Replace Travis CI with Github Actions (#1926)
Travis stopped working again and no answers from support
2021-01-11 22:39:15 +05:00
FiveFilters.org
f927781750 [TwitterBridge] Add option to hide pinned tweet (#1908) 2021-01-10 13:50:06 +05:00
Jacques Heunis
e128ce807a [ItchioBridge] Add bridge (#1918) 2021-01-10 13:30:12 +05:00
Derrick Lee
6b870f0c3e [TwitterBridge] Fix username matching to be case insensitive with noretweet option (#1924) 2021-01-10 13:19:38 +05:00
JimDog546
5ed161943c [InstagramBridge] Remove redundant data collection for sidecar and video (#1920)
getInstagramSidecarData and getInstagramVideoData were unnecessarily calling getSinglePostData to retrieve data already present in collectData's call of getInstagramJSON. getSinglePostData sometimes doesn't retrieve data properly resulting in incomplete post information. Since the information needed is already present, pass it from collectData instead, eliminating the redundant data collection and improving reliability.
2021-01-10 13:14:58 +05:00
Corentin Garcia
1edec1aa45 [GenshinImpactBridge] Add bridge (#1912) 2020-12-30 23:30:02 +05:00
dawidsowa
3c285d50ec [RedditBridge] Rewrite to use JSON (#1781) 2020-12-23 22:42:15 +05:00
Clemens Neubauer
3aae00b56a [HDWallpapersBridge] Fix URLs (#1892) 2020-12-23 22:19:32 +05:00
sysadminstory
21798e8228 [ZoneTelechargement] Add support for Streaming links (#1858) 2020-12-23 22:13:10 +05:00
Eugene Molotov
3226a5e31e [core] Use more correct text to indicate errored response from upstream 2020-12-23 17:49:11 +05:00
Eugene Molotov
59bbc9d2e7 [VkBridge] Several improvements (#1802)
* Improved post author computation
* Show repost sources
* Handle second copy quote
* Fixed incorrect image uri's
2020-12-18 07:58:51 +05:00
Eugene Molotov
2ddd357a62 [JS] Fixed TypeError: textValue is null 2020-12-18 07:51:33 +05:00
Devon Hess
c302bca1e6 [IKWYDBridge] New bridge (#1874) 2020-12-13 16:08:25 +05:00
t0stiman
0b3082609d [RaceDepartmentBridge] added a bridge for racedepartment.com (#1681) 2020-12-13 15:50:54 +05:00
Lyra
810a2503c9 [core] Add configuration for bridges, allowing private bridges (#1343) 2020-12-12 21:05:22 +05:00
Eugene Molotov
56b2c516e4 [InstagramBridge] pauder is defacto not a maintainer of InstagramBridge 2020-12-09 10:47:48 +05:00
Eugene Molotov
fc81bed717 [BridgeAbstract] Correct getIcon method 2020-12-07 21:08:58 +04:00
Tobias Alexander Franke
56eb829a66 [EconomistBridge] Fixes for fetching new page structure (#1836) 2020-11-29 15:31:20 +05:00
Alex Kirk
7705d097e3 [SkimfeedBridge] Add parameter detection (#1877) 2020-11-29 15:28:07 +05:00
Peter Dave Hello
be9df41e07 [Dockerfile] Clean up apt cache, make image smaller (#1880) 2020-11-29 15:18:20 +05:00
Joseph
1e75f9d3d5 [ReporterreBridge + KernelBugTrackerBridge + BastaBridge] Use defaultLinkTo() (#1862) 2020-11-23 23:49:25 +05:00
Eugene Molotov
0755181555 [CeskaTelevizeBridge] Remove executable flag from bridge file 2020-11-19 16:19:46 +05:00
Joseph
e6c73a1fe3 [FlickrBridge] Add filter by media and sort by options (#1758) 2020-11-16 22:33:48 +05:00
David Pedersen
5729e069e9 [AppleMusicBridge] Use title from website (#1855) 2020-11-16 22:15:39 +05:00
David Pedersen
0b494d9c0e [AmazonPriceTrackerBridge] Add support for Swedish Amazon (#1856) 2020-11-16 22:13:23 +05:00
Eugene Molotov
c855d5089f Revert "[TelegramBridge] Prevent double encoding entities (#1182)"
This reverts commit 5e2f0fb626.
2020-11-11 23:31:24 +05:00
dawidsowa
6baf64f29e [Rule34pahealBridge] Use full size image (#1775) 2020-11-11 23:28:12 +05:00
Alex Balgavy
7d4b76be99 [SeznamZpravyBridge] New bridge (#1806) 2020-11-11 22:39:34 +05:00
Eugene Molotov
ff50e4918c Bump version to dev.2020-11-10 2020-11-10 16:26:08 +05:00
Eugene Molotov
8e4d6d8fdb [README] Update list of contributors 2020-11-10 16:24:58 +05:00
ayacoo
cc548b16a8 [HeiseBridge] Check for article (#1790) 2020-11-10 11:14:09 +05:00
Roman Remizov
b66026e241 [TwitterBridge] Add support for querying by list ID (#1834) 2020-11-10 11:12:02 +05:00
Joseph
a23d4bd0e6 [BrutBridge] Add support for Spain edition and health category (#1833) 2020-11-09 18:14:13 +05:00
ymeister
bde4159a9e [README] Add "filter" to Requirements (#1827)
This extension is required by filter_var()
2020-11-08 12:25:06 +05:00
Niehztog
3ad138026d [BridgeXPathAbstract + BlizzardNewsBridge + XPathBridge] Add new abstract class + two example implementations (#1671) 2020-11-08 12:22:41 +05:00
Joshua Coales
d05a8b79fe [contents.php] Fix return type hints (#1824) 2020-11-08 12:19:18 +05:00
Joshua Coales
efe32aad22 [FacebookBridge] Permalink fix (#1838)
Also removing timestamp which does not work
2020-11-06 18:43:10 +05:00
Petr Kolář
0655b3cb39 [MallTvBridge] New bridge (#1819) 2020-10-31 22:05:13 +05:00
csisoap
59082368c7 [BleepingComputerBridge] New bridge (#1815) 2020-10-31 22:01:19 +05:00
msch
c8b2c1bf74 [OpenwrtSecurityBridge] Add new bridge (#1812) 2020-10-31 21:57:29 +05:00
Roliga
b48bc77c22 [TwitchBridge] Switch to unofficial GraphQL API (#1829)
* [TwitchBridge] Switch to unofficial GraphQL API

The GraphQL API that the twitch.tv website uses has a lot more
information available than the official APIs. Hopefully it'll be stable.
2020-10-30 13:50:36 +00:00
Joshua Coales
6af87b2f32 [FacebookBridge] Use touch.facebook.com for groups (#1817) 2020-10-29 08:42:49 +05:00
Eugene Molotov
93cdf5e342 [core] Fixed passive XSS vulnerability
Reference: https://www.openbugbounty.org/reports/1140367/
2020-10-26 15:08:11 +05:00
Joseph
164b407f28 [BridgeCard] Fix parameter layout issue (#1816)
Fixes parameter layout issue on small screens.
2020-10-26 12:11:58 +05:00
ORelio
2714c3d816 [WordPress] Limit feed to 20 items (#1801) 2020-10-21 14:59:04 +05:00
Eugene Molotov
364b5282a3 [GoogleSearch] Use other class for content retreiving (#1803) 2020-10-19 16:22:37 +05:00
Eugene Molotov
5e4f3c351e [NineGagBridge] Lint previous commit 2020-10-15 14:18:46 +05:00
Gregor Santner
a332a5a414 [NineGagBridge] In post URI replace scheme from "http" to "https" 2020-10-15 14:12:54 +05:00
Joshua Coales
45e2f385b3 [FacebookBridge] Handle mobile links and unify host validation (#1789) 2020-10-15 14:08:03 +05:00
Joseph
0a1ff10a52 [KoreusBridge + VarietyBridge] Use HTTPS when fetching feedburner feeds (#1797) 2020-10-15 13:03:51 +05:00
Eugene Molotov
645a8f62c6 [.travis] Fix several phpcs and phpunit errors (#1799) 2020-10-15 12:53:19 +05:00
ORelio
64ec488f70 [ZDNet] Fix article layout (#1793) 2020-10-13 21:46:58 +05:00
Petr Kolář
7b6ff78623 [CeskaTelevizeBridge] Add New bridge (#1784) 2020-10-12 11:35:06 +05:00
Corentin Garcia
82acbbb421 [DribbbleBridge] Fix picture parsing (#1787) 2020-10-10 00:46:40 +05:00
Ololbu
84d5daaa03 [FicbookBridge] Add getName implementation (#1771) 2020-10-10 00:39:35 +05:00
Joseph
712f60e910 [HeiseBridge] Fix multi-page article fetching (#1767)
Fixes multi-page article fetching by adding '&seite=all' to  article URL.
2020-10-09 23:55:28 +05:00
Alexander
55015f80cf [AlbionOnlineBridge] New bridge (#1769) 2020-10-09 23:48:40 +05:00
csisoap
f90c6b5bb9 [NasaApodBridge] Fix broken image link (#1778) 2020-10-09 23:33:54 +05:00
Corentin Garcia
ff98efe8dc [core] Use Parsedown for Markdown parsing (#1783) 2020-10-09 23:29:02 +05:00
Corentin Garcia
fe166d0216 [NasaApodBridge] Fix header being parsed as item (#1586) 2020-10-07 11:16:26 +05:00
triatic
d3455dd18a [TwitterBridge] Optimise regular expression code (#1768)
* [TwitterBridge] Optimise regular expression code

Optimise regular expression search code so adding new URLs is cleaner
2020-10-05 12:07:39 +05:00
ORelio
47dc26c775 [NextINpact] Fix subtitle extraction in #LeBrief (#1780)
The bridge was taking another article abstract as subtitle for #LeBrief articles
2020-10-05 11:57:13 +05:00
Ololbu
3df2de4c6f [FicbookBridge] Fix data getting and months (#1765) 2020-09-28 14:02:40 +05:00
sarnd
01985b7af7 [TwitterBridge] URL to js file with apikey changed again (#1764) 2020-09-28 10:01:37 +05:00
Joseph
80cc88ba78 [SoundcloudBridge] Fix bridge not returning tracks (#1757)
+ Use artwork for enclosure
2020-09-25 11:43:12 +05:00
Christian Schabesberger
2bb99c4448 [NordbayernBridge] Fix images and newsblock order (#1741) 2020-09-18 10:13:31 +05:00
Jason Ghent
3a29347e60 [ParameterValidator] Ignore cache-busting param (#1723) 2020-09-14 14:01:55 +05:00
Alexander
d299adb827 [EpicgamesBridge] Add pinned posts to list (#1736) 2020-09-14 13:14:35 +05:00
Michael Bemmerl
cf606a3a6b [OtrkeyFinderBridge] Add bridge for otrkeyfinder.com (#1712) 2020-09-11 11:48:03 +05:00
Eugene Molotov
6c244f4d9b [TwitterBridge] Skip advertisment tweets (#1673) 2020-09-11 11:44:28 +05:00
AxorPL
d6f277d029 [WorldCosplayBridge] Add new bridge (#1732) 2020-09-09 17:11:19 +05:00
sysadminstory
ab8e89a97f [AllocineFRBridge] Update CSS class (#1585)
Website has change one CSS class : updated the bridge to allow parsing.
2020-09-08 10:55:21 +05:00
Joseph
747bb6ad9c [WosckerBridge] Add bridge (#1643) 2020-09-03 11:18:15 +05:00
Joseph
d33e090fe1 [MastodonBridge] Update feed URL format (#1718)
Changes feed URL from `https://instance/users/username.atom` to `https://instance/@username.rss`.
2020-09-03 10:49:19 +05:00
Christian Jonak
fec52418d5 [FM4Bridge] Add new bridge for FM4 news page (#1719) 2020-09-03 10:46:35 +05:00
ggiessen
bb51a0d212 [MarktplaatsBridge] Improvements (#1722)
- sometimes $listing->imageUrls is empty so moved after the if statement on line 91 
- added price and location info
- added function getName
2020-09-03 10:44:32 +05:00
Bob
68dd2d745f [InstagramBridge] Change TAG_QUERY_HASH (#1727) 2020-09-02 11:02:34 +05:00
ORelio
46abc18e87 [Anidex] Fix content retrieval (#1693)
Anidex uses two separate domains, anidex.info and anidex.moe
anidex.info has ddos-guard so we need to request
anidex.moe with Host header set to anidex.info
2020-08-31 22:04:56 +05:00
ORelio
e00bbe353f * [ReleasesSwitch] Switch scene releases (#1694)
Separate bridge from Releases3DS that just has a different URL.
Inherits from Releases3DS so both bridges need to be present.

*  [Releases3DS] Fix PHP notices related to IGN
2020-08-31 22:02:25 +05:00
somini
c21a805cb4 [DiarioDeNoticiasBridge]: New Bridge (#1717) 2020-08-27 10:38:51 +05:00
Simon Alberny
3b36c413e5 [MondeDiplo] Switched to HTTPS + Title and content updated (#1714) 2020-08-27 10:28:59 +05:00
ggiessen
94576c3053 [MarktplaatsBridge] 'https:' added to img src url (#1713) 2020-08-24 10:30:59 +05:00
ronansalmon
25cff9c07b [TwitterBridge] Convert plain text URLs into HTML hyperlinks (#1627) 2020-08-21 17:55:11 +05:00
ORelio
07c71b3b36 [NextINpact] Upgrade for NextINpact v7 (#1708) 2020-08-20 17:49:26 +05:00
Alexander
859053ef7a [EpicgamesBridge] New bridge (#1709) 2020-08-20 10:36:11 +05:00
triatic
73287f536b [TwitterBridge] Add retweeter to retweeted tweets (#1679) 2020-08-20 10:00:27 +05:00
sysadminstory
0b1e592a5e [ZoneTelechargement] Update URL (#1710)
The bridge now shows links to the new URL.

It keeps the old one internally to bypass the Robot protection on the
new URL.
2020-08-19 17:35:19 +05:00
jannyba
ef54a78430 [InstagramBridge] Fix "Skip reviews" checkbox description (#1702) 2020-08-16 11:23:48 +05:00
Eugene Molotov
4b8c3b9d36 [Multi] Minor improvements for my bridges (#1507)
* [DarkReading] Hide dummy articles

* [FuturaSciences] Strip inline scripts from content

* [FeedExpander] Fix PHP notice on missing uri field

(guid is valid uri AND item uri is not valid)
 => (guid is valid uri AND item uri is empty or not valid)

* [NextInpact] Fix subtitle extraction

* [Markdown] Fix images with empty replacement text

* [TheHackerNews] Fix Author name cleanup

* [LeMondeInformatique] Remove encoding conversion

Was previously needed due to actual encoding on the page
being inconsistent with encoding specified in <meta> tag

* [AnimeUltime] Remove encoding conversion

Was previously needed due to encoding on the page being incorrect

* [FuturaSciences] Fix content extraction

* [FuturaSciences] Fix unneeded unset()

* [GBAtemp] Fix tutorial mode URL extraction

* [GBAtemp] Fix tutorial mode Title extraction
2020-08-14 10:30:31 +05:00
ORelio
c642652fea [GBAtemp] Fix tutorial mode Title extraction 2020-08-12 20:08:24 +02:00
ORelio
f0e6298cab [GBAtemp] Fix tutorial mode URL extraction 2020-08-12 20:08:24 +02:00
ORelio
45e247b9d0 [FuturaSciences] Fix unneeded unset() 2020-08-12 20:08:24 +02:00
ORelio
66a009b8fb [FuturaSciences] Fix content extraction 2020-08-12 20:08:24 +02:00
ORelio
efd1abfab1 [AnimeUltime] Remove encoding conversion
Was previously needed due to encoding on the page being incorrect
2020-08-12 20:08:24 +02:00
ORelio
8b173b8874 [LeMondeInformatique] Remove encoding conversion
Was previously needed due to actual encoding on the page
being inconsistent with encoding specified in <meta> tag
2020-08-12 20:08:24 +02:00
ORelio
90e9c9962a [TheHackerNews] Fix Author name cleanup 2020-08-12 20:08:24 +02:00
ORelio
01cc32a0cc [Markdown] Fix images with empty replacement text 2020-08-12 20:08:24 +02:00
Joseph
dc36b425cd [DevToBridge] Fix bridge (#1699)
Fixes full article option not working
2020-08-12 23:07:53 +05:00
triatic
268ddf1382 [TwitterBridge] URL to js file with apikey changed (#1698)
Fixes #1697
2020-08-12 10:32:34 +05:00
ORelio
8144488a9e [FeedExpander] Fix PHP notice on missing uri field
(guid is valid uri AND item uri is not valid)
 => (guid is valid uri AND item uri is empty or not valid)
2020-08-11 14:01:44 +02:00
ORelio
062dd7f8a5 [FuturaSciences] Strip inline scripts from content 2020-08-11 14:01:44 +02:00
Nemo
be089702f0 [docker] Install memcached PHP extension from PECL (#1473)
- Adapted from https://stackoverflow.com/a/41575677
- Tested by running `php -m`
- SimpleXML now comes pre-installed with PHP, so removed the extra step
- Adds --no-install-recommends

Signed-off-by: Nemo <me@captnemo.in>
2020-08-07 18:21:28 +05:00
sarnd
c71fad4a4a [TwitterBridge] URL to js file with apikey changed. (#1686)
Twitter has changed URL scheme back again (see PR#1647 / commit 78298385d0)

This patch will try both URL schemes now and throw a specific error when neither works
2020-08-06 10:22:17 +05:00
Eugene Molotov
5be251a66e [TwitterBridge] Fetch latest tweets on hashtag or keyword query (#1674) 2020-08-02 19:40:41 +05:00
Eugene Molotov
7709b8d662 [VkBridge] Correct fallback behavior, when trying to get direct video links (#1670) 2020-07-31 15:29:18 +05:00
triatic
f5916a2f74 [TwitterBridge] apikey fetched every time (#1663)
The apikey is fetched every time because $data is not an array. Update the condition to expire the api key at the same time as the guest token.
2020-07-30 09:54:16 +05:00
Eugene Molotov
a33088ca99 [GoogleSearch] Correct parsing uri of search result (#1601) 2020-07-27 11:44:07 +05:00
somini
78facbcb83 [TwitterBridge] Fix noretweet for users (#1608)
This also removes spurious retweets.
2020-07-26 11:26:39 +05:00
Thomas
d5a75a2545 [DribbbleBridge] regex fix and CSS selector update (#1657)
* [DribbbleBridge] Fixed regular expressions for quote replacement in JSON (previously invalid JSON was created if a property value contained colons or single quotes). Also updated two CSS selectors as Dribbble's HTML has changed.

* [DribbbleBridge] Added fix for relative dates in JSON

* [DribbbleBridge] Removed redundant whitespaces
2020-07-25 08:58:42 +05:00
triatic
25698d182c [TwitterBridge] Remove unused variable 2020-07-24 13:13:21 +05:00
Corentin Garcia
9e74cc64ed [RainbowSixSiegeBridge] Fix bridge (#1587) 2020-07-24 12:56:41 +05:00
sarnd
78298385d0 [TwitterBridge] guest token is returned via body again. (#1647)
* [TwitterBridge] guest token is returned via body again. This change will try to search fot token inside header and fallback to body

* Twitter changed the URL scheme for the API
2020-07-24 12:52:27 +05:00
somini
976445b490 Improve Soundcloud bridge (#1500)
* [SoundcloudBridge] Add playlist support, migrate to `api-v2`
2020-07-05 19:49:46 +02:00
triatic
3ad126cdf2 [core] Add headers to file_get_contents (#1623)
Add response headers to file_get_contents() method. Headers are used by some bridges.
2020-06-25 12:22:05 +02:00
sarnd
e87b868307 [TwitterBridge] Fix issue #1621 @<twitter_user> failed with error 429 (#1622)
* [TwitterBridge] Fix issue #1621 @<twitter_user> failed with error 429
2020-06-25 12:21:48 +02:00
triatic
23c61f5f84 [TwitterBridge] Expire guest token by time (#1606)
* [TwitterBridge] Expire guest token by time

In addition to fetching a new guest token after 100 uses, also expire token after 5 minutes (configurable).
2020-06-23 15:14:50 +02:00
somini
22a01f1093 [Twitter] Fix Twitter bridge images and add other media types (#1595)
* Keep old URI structure

Use the username, not the user ID.

* Fix Twitter bridge images

Credit to @kinoushe

See https://github.com/RSS-Bridge/rss-bridge/issues/1562#issuecomment-639393175

* Include Videos and "Animated GIF" as twit enclosures

Credit to @kinoushe for digging into the API docs.

https://github.com/RSS-Bridge/rss-bridge/issues/1562#issuecomment-640320688

* Calculate the highest bitrate video

Include that on the enclosure.

* Appease linter

* Appease linter, again

* Remove surrounding link from videos

Add it on a smaller link besides it.

See
https://github.com/RSS-Bridge/rss-bridge/pull/1595#issuecomment-640989208

* Include video poster on the enclosures.
2020-06-10 22:39:36 +02:00
Park0
98ff5a095c [Marktplaats] New Bridge (#1575) 2020-06-09 20:21:34 +02:00
Eugene Molotov
e4c4ae8245 [MemcachedCache] loadData now returns null instead of false (#1592)
FileCache and SQLiteCache returns null on cache miss. This is important if using strict comparing (for example when using "===")
2020-06-08 11:27:19 +02:00
Lyra
124631df73 [TwitterBridge] Fix caching policy, usernames as well as images 2020-06-08 11:18:24 +02:00
Lyra
06891ae35f [TwitterBridge] Fix the bridge using a brand new API 2020-06-05 10:17:53 +02:00
Michael Bemmerl
c4422bdbb5 [Core] Fix notice of undefined offset when in detached HEAD state. (#1569) 2020-05-27 23:08:06 +02:00
floviolleau
a1dd98ff82 [LesJoiesDuCodeBridge] Fix items not loading 2020-05-27 23:04:43 +02:00
floviolleau
25f0d3b877 [TheCodingLoveBridge] Fix not loading items (#1577) 2020-05-27 23:04:03 +02:00
Damien Calesse
9a66227a79 [SensCritique] Fix search display (#1567)
- Remove movies search. It appears the website changed their movies
  displays and data cannot be easily extracted for now.
- Fix some errors on items without proper description and/or original
  title.
2020-05-20 21:52:37 +02:00
Sandro
8047041963 [Core] Include Media RSS namespace for Atom feeds
Include Media RSS namespace for Atom feeds
Fix #1511
Fix #1499
2020-05-19 10:00:12 +02:00
Joseph
fa74d3728b [GizmodoBridge] Fix bridge (#1538)
* [GizmodoBridge] Update bridge
2020-05-17 20:35:34 +02:00
Fanch
8233497611 [AirBreizhBridge] Add new bridge (#1544)
* [AirBreizhBridge] Add new bridge
2020-05-17 20:33:14 +02:00
Joseph
71745116e1 [MozillaBugTrackerBridge] Fix bridge (#1550)
* [MozillaBugTrackerBridge] Fix bridge
2020-05-17 20:33:01 +02:00
Paroleen
36fc4822dd [UnraidCommunityApplicationsBridge] Add new bridge (#1534)
* [UnraidCommunityApplicationsBridge] Add new bridge
2020-05-17 20:22:04 +02:00
Eugene Molotov
868d3f600d [VkBridge] Fix one letter bug on titles (#1555) 2020-05-17 20:21:37 +02:00
sysadminstory
f4affe1833 [AuoJMBridge] Follow Website change (#1527)
* [AuoJMBridge] Follow Website change
2020-05-17 20:05:04 +02:00
Joseph
63a4db7e86 [DownDetectorBridge] Fix bridge (#1528) 2020-05-17 20:04:37 +02:00
Joseph
f48909b84e [ASRockNewsBridge] Add Bridge (#1526)
* [ASRockNewsBridge] Add Bridge
2020-05-17 20:00:52 +02:00
Paroleen
ca88096f1f [AwwwardsBridge] New bridge (#1524)
[AwwwardsBridge] New bridge (#1524)
2020-05-17 19:58:19 +02:00
Lyra
1044952987 [MediapartBlogsBridge] Lint
[MediapartBlogsBridge] Lint
2020-05-17 19:49:00 +02:00
Eugene Molotov
119f4bdec5 [MediapartBlogsBridge] Lint 2020-05-10 17:35:21 +05:00
Lyra
e617d9f728 [MediapartBlogsBridge]: New Bridge
[MediapartBlogsBridge]: New Bridge
2020-04-03 10:02:47 +02:00
Lyra
5a43db4fb5 [FolhaDeSaoPauloBridge]: Improvements
[FolhaDeSaoPauloBridge]: Improvements
2020-04-03 09:53:42 +02:00
Lyra
badb5313b7 [NordBayernBridge] Add bridge (#1513)
* add Nordbayern bridge
2020-04-03 09:52:02 +02:00
Lyra
5eeda8dd52 Merge pull request #1515 from Dreckiger-Dan/patch-1
[HeiseBridge] add TechStage support
2020-04-03 09:50:08 +02:00
Lyra
413ae3cef6 [GithubTrendingBridge] Add bridge (#1492)
* Added GithubTrendingBridge
2020-04-03 09:48:31 +02:00
Christian Schabesberger
604d527ac7 add nordbayern bridge
fix intending
2020-04-02 12:44:10 +02:00
Dreckiger-Dan
cccd390b0f [HeiseBridge] add TechStage support 2020-03-31 23:47:57 +02:00
somini
223337d62d [FolhaDeSaoPauloBridge]: Improve URL
Remove the redirection.
2020-03-31 02:34:38 +01:00
somini
066e42e99a [FolhaDeSaoPauloBridge]: Improve HTML 2020-03-31 02:32:15 +01:00
Kirill Kotikov
fbfc82b0b7 Revert feed title 2020-03-26 21:37:19 +03:00
ORelio
00dd81a8aa [DarkReading] Hide dummy articles 2020-03-25 20:40:17 +01:00
somini
e0ac9972ee [MediapartBlogsBridge]: New Bridge
Fix #1468
2020-03-25 19:02:09 +00:00
Kirill Kotikov
f2de5aecc7 Change feed title 2020-03-24 19:07:23 +03:00
Kirill Kotikov
0fd7021030 Change cache time to 24hr (daily update time) 2020-03-23 10:35:02 +03:00
Kirill Kotikov
3ec32bb6c2 Fix title if language not set 2020-03-22 21:43:37 +03:00
Kirill Kotikov
ec7ef8f502 Update GithubTrendingBridge.php 2020-03-21 05:07:38 +03:00
Kirill Kotikov
7b73f3217f Fix page request 2020-03-21 05:01:45 +03:00
Kirill Kotikov
7c71377af0 Add additional languages + fix issues 2020-03-20 16:34:42 +03:00
Kirill Kotikov
c2559ff71f Add sdfsf 2020-03-16 19:25:28 +03:00
John Corser
366d2d66b3 [RobinhoodSnacks] Add bridge for Robinhood Snacks (#1460) 2020-02-26 23:32:57 +01:00
Lyra
7b63da522f [InstagramBridge] Use lowercase comparison when looking up user pk 2020-02-26 22:35:44 +01:00
Lyra
0705a2e7bb Bump version to dev.2020-02-26 2020-02-26 22:24:20 +01:00
Lyra
84616f53bf Update contributors 2020-02-26 22:23:30 +01:00
Eugene Molotov
a981450ae0 [Dockerfile] Build memcached extension (#1415) 2020-02-26 22:16:46 +01:00
somini
d39741c296 [GithubIssueBridgeIssue] Fix bridge (#1453)
* fix bridge according to website evolution
2020-02-26 22:15:50 +01:00
Lorenzo Stanco
3179c1e884 [InstagramBridge] Fixed item thumb on video entries (#1387) 2020-02-26 22:13:40 +01:00
sysadminstory
c9e5f6c9dd [AllocineFRBridge] Update Show List and parsing (#1407)
* [AllocineFRBridge] Update Show List and parsing
2020-02-26 22:12:25 +01:00
Julien Desgats
6b6974d115 [NewOnNetflix] Add new bridge (#1408) 2020-02-26 22:11:54 +01:00
Anchit Bajaj
96e58d4c94 Add bridge for Phoronix (#1412) 2020-02-26 22:10:54 +01:00
Anchit Bajaj
f0363ba03b [PcGamerBridge] - Add all articles, full content and images (#1420) 2020-02-26 22:10:09 +01:00
somini
90147fc45c [FirstLookMediaTech]: New Bridge (#1438) 2020-02-26 22:08:14 +01:00
John Corser
a3b4bd2d08 [DaveRamseyBlogBridge] Add new bridge (#1459) 2020-02-26 22:05:55 +01:00
St. John Johnson
e102353ab8 [GoComics] Update to new website structure (#1464)
GoComics.com has updated their website.  The image location is now a
data attribute in a div.
2020-02-26 21:56:52 +01:00
Joseph
a54eb88ee1 [DevToBridge] Fix bridge & add getName() (#1470) 2020-02-26 21:56:03 +01:00
somini
1584636e5b TinyLetter: New Bridge (#1469)
* TinyLetter: New Bridge
2020-02-26 21:50:25 +01:00
Joseph
fe83d763a3 [PornhubBridge] Fix travis issues (#1471)
* [PornhubBridge] Fix travis issues
2020-02-26 21:34:46 +01:00
Mitsukarenai
480694e819 [PornhubBridge] Add bridge 2020-02-15 00:03:29 +01:00
Tyler Kenney
8697e1e1a2 [RoosterTeethBridge] Add a new bridge (#1450)
* Added RoosterTeethBridge
2020-02-10 16:57:08 +01:00
Binnette
1ab7e493a8 [DonnonsBridge] Add a new bridge (#1441) 2020-02-10 16:56:40 +01:00
86423355844265459587182778
e5303efba3 [SoundcloudBridge] Fix returned URL and title (#1449) 2020-02-07 16:16:55 +01:00
Joseph
5bd07723ad [ScribdBridge] Add bridge (#1391) 2020-02-04 17:26:34 +01:00
Anchit Bajaj
00dbde2c24 [IGNBridge] Removed Ugly Nonworking Widgets (#1413) 2020-02-04 17:25:56 +01:00
floviolleau
a00e75b71c [AtmoOccitanieBridge] Add new bridge for air quality in cities in Occitanie (#1422)
* Add new bridge for Air Quality in cities supported by Atmo Occitanie
2020-02-04 17:24:42 +01:00
floviolleau
f040e4dc9c [AtmoNouvelleAquitaine] Change description (#1423)
* [AtmoNouvelleAquitaine] Change description
2020-02-04 17:22:42 +01:00
sysadminstory
182e9e7b41 [ZoneTelechargement] Update URL (#1425)
Website changed again his URL
2020-02-04 17:21:02 +01:00
somini
275662b8d4 [FolhaDeSaoPaulo]: Add new Bridge (#1426)
* [FolhaDeSaoPaulo]: Add new Bridge
2020-02-04 17:19:39 +01:00
Antoine Turmel
f52eb43f8c Update GithubSearchBridge.php (#1431)
Fixes #1430
2020-01-31 15:01:46 +01:00
sysadminstory
2450f80823 [ExtremeDownloadBridge] Update URL (#1429)
Website URL has changed again !
2020-01-31 15:00:17 +01:00
Corentin Garcia
45287e6853 [RainbowSixSiegeBridge] Fix bridge (#1433) 2020-01-31 14:51:59 +01:00
Eugene Molotov
830f57f607 [TwitterBridge] Use IE's user-agent (#1442)
Twitter will return pages with legacy design and frontend code, which bridge can deal with
2020-01-31 14:36:25 +01:00
Eugene Molotov
6a90a9d33f phpcs: fix new sudden violations (#1443) 2020-01-31 14:30:31 +01:00
Eugene Molotov
46b9879c08 [VkBridge] Correct post date calculating (#1417)
* [VkBridge] Correct post date calculating

Before this commit, post dates from december past year were
calculated as december current year.
2020-01-16 12:00:10 +01:00
Mitsukarenai
1343dbe97a [index] Bump spoofed user-agent version 2020-01-15 21:36:12 +01:00
Nono
2175a4d08b [MozillaSecurityBridge] source has been modified (#1394)
adjustement following source change
2020-01-10 14:22:58 +01:00
Joe Digilio
ad661c4c91 [RedditBridge] Fix typo prevents bridge from working (#1383) 2019-12-05 18:07:50 +01:00
Grégory T
ba8c4623ed [DisplayAction] Fix function call on a member (add ->) (#1379) 2019-12-04 18:34:26 +01:00
logmanoriginal
ba43c87952 [RevolutBridge] Remove bridge
An official RSS feed is available at https://blog.revolut.com/rss/

Note that there is also an invisible "RSS" button next to the Facebook
and Twitter icons at the menu bar.

References #1321
2019-12-04 18:23:13 +01:00
Grégory T
595b87946d [TorrentGalaxyBridge] Add new bridge (#1378) 2019-12-02 20:31:50 +01:00
logmanoriginal
99d4e1a43d Bump version to dev.2019-12-01 2019-12-01 13:40:17 +01:00
logmanoriginal
477de4e2df Bump version to 2019-12-01 2019-12-01 13:34:09 +01:00
logmanoriginal
246470da18 [README] Update list of contributors 2019-12-01 13:33:03 +01:00
LogMANOriginal
df9f7eb778 [FacebookBridge] Fix permalink issue (#1358)
Facebook has changed their strategy regarding permalinks, which
now include lots of unnecessary target data. Fortunately it also
contains the unique story id which we can utilize as URI.
2019-12-01 13:24:11 +01:00
David
375831f516 [NineGagBridge] Add filter option for animated content (#1374) 2019-12-01 13:07:25 +01:00
Roliga
e518936be7 [SoundcloudBridge] Automatically acquire client_id (#1375)
Also some slight refactoring, as well as adding Roliga as maintainer.
2019-12-01 12:42:53 +01:00
Jacob Mansfield
583dfb4958 [TheWhiteboard] Create new bridge for The Whiteboard (#1327) 2019-12-01 11:30:16 +01:00
Jacob Mansfield
2de45b163e [FurAffinityUser] Add new bridge (#1326) 2019-12-01 11:27:41 +01:00
Shuto Yano
48b0164676 [InstagramBridge] Fix instagram GraphSidecar output and Video embedding (#1361)
* [InstagramBridge] Fix GraphSidecar output

Fix following issues which related to output of the GraphSidecar type posts.
- The GraphSidecar post's media wasn't outputted except for first picture when searching by hashtag or location
- Video didn't embedded
NOTE:
The function getInstagramStory() which was called when the post type is GraphSidecar didn't seem to work just as one intended.
Because the web request called in that function is just to get the media of single post, NOT to get the media of Story.
But I don't have any idea to solve #694, so it seems be better to rename these function and member variable properly.
2019-12-01 11:25:20 +01:00
somini
4b3c3c58d2 [DiárioDoAlentejo] Add new bridge (#1360) 2019-12-01 11:18:45 +01:00
somini
60768b4885 [DisplayAction] Don't return redirect error codes (#1359)
This might lead to redirect loops.

See
https://github.com/RSS-Bridge/rss-bridge/pull/1071#issuecomment-515632848

Cherry-picked from eb21d6f.
2019-12-01 11:13:57 +01:00
Eugene Molotov
02dd778124 [VkBridge] Save internal links in posts and get hashtags before making item (#1363) 2019-11-18 10:51:35 +01:00
Eugene Molotov
5b63121e92 [VkBridge] Change access token (#1357)
Previous access token was revoked
2019-11-09 18:51:16 +01:00
St. John Johnson
49019a843f [ComicsKingdomBridge] Add new bridge (#1353) 2019-11-09 18:50:08 +01:00
Roliga
d65714fa47 [HtmlFormat] Add syndication links (#1348)
Adds <link> elements for each additional output format in the <head> of
HTML format output to allow RSS readers to find the actual feeds
directly from the HTML page.
2019-11-09 18:43:21 +01:00
Joseph
8161829ad5 [OpenwhydBridge] Add new bridge (#1338)
Rename WhydBridge to OpenwhydBridge
2019-11-09 18:36:50 +01:00
Nemo
7f35fc9f6b [AppleAppStoreBridge] Add new bridge (#1316)
This bridge allows you to follow iOS/iPad application updates
directly over RSS. This allows you to get notified of an application
update even if you don't have a iDevice. This may be useful in cases
where you want to be notified of a competitor's application for eg.

I built this after I got tired of waiting for WhatsApp to push their
security release on iOS. It shows up on the AppStore 7 days later
2019-11-09 18:28:00 +01:00
logmanoriginal
3bc8c9468a phpcs: Always use long array syntax
Most of the code in RSS-Bridge uses the long array syntax.
This commit adds a check to enforce using this syntax over
the short array syntax.

All failures have been fixed.
2019-11-01 18:06:55 +01:00
logmanoriginal
1df3598a74 [Dockerfile] Drop minimum security level back to TLS 1.0
Debian increased the minimum security level for OpenSSL from TLS 1.0
to TLS 1.2 [1] which also affects the Debian-based PHP image for Docker.

This change can break some bridges which have to connect to servers with
lower security level. Since all browsers still connect to these servers,
so should RSS-Bridge.

Note that according to [2] Mozilla, Firefox, Microsoft, Google and Apple
plan to increase the minimum security level to TLS 1.2 around March 2020.
At this time RSS-Bridge should follow the browser changes.

This commit updates the Dockerfile to automatically drop the minimum
security level back to TLS 1.0.

Based on the solution provided by @theScrabi in #1318

[1] https://wiki.debian.org/ContinuousIntegration/TriagingTips/openssl-1.1.1
[2] 553fc8e61f/debian/libssl1.1.NEWS
2019-11-01 17:12:45 +01:00
logmanoriginal
5f64fe2516 [BridgeAbstract] Fix broken assignment of defaultValue
setInputs() currently looks if the global array defines a 'value'
for a given parameter, but that isn't supported by the API. It
needs to be 'defaultValue'.
2019-11-01 15:29:16 +01:00
logmanoriginal
50eee7e7b3 [KununuBridge] Add feed item limit
This bridge currently takes a very long time to process
all news items on the page, when in many cases only one
or two had been added since the last check.

This commit adds a new parameter 'limit', which defines
the maximum number of items to add to the feed. This is
an optional paramter that defaults to 3.
2019-11-01 15:27:35 +01:00
logmanoriginal
c0df9815c7 [DesoutterBridge] Add feed item limit
This bridge currently takes a very long time to process
all news items on the page, when in many cases only one
or two had been added since the last check.

This commit adds a new parameter 'limit', which defines
the maximum number of items to add to the feed. This is
an optional paramter that defaults to 3.
2019-11-01 15:07:25 +01:00
Léo Maradan
46d5895d1d [RedditBridge] Add new bridge (#1213) 2019-11-01 13:54:03 +01:00
Anchit Bajaj
7c16aaf303 [VarietyBridge] Add new bridge (#1307) 2019-11-01 13:48:09 +01:00
LogMANOriginal
cdc1d9c9ba action: Add action to check bridge connectivity (#1147)
* action: Add action to check bridge connectivity

It is currently not simply possible to check if the remote
server for a bridge is reachable or not, which means some
of the bridges might no longer work because the server is
no longer on the internet.

In order to find those bridges we can either check each
bridge individually (which takes a lot of effort), or use
an automated script to do this for us.

If a server is no longer reachable it could mean that it is
temporarily unavailable, or shutdown permanently. The results
of this script will at least help identifying such servers.

* [Connectivity] Use Bootstrap container to properly display contents

* [Connectivity] Limit connectivity checks to debug mode

Connectivity checks take a long time to execute and can require a lot
of bandwidth. Therefore, administrators should be able to determine
when and who is able to utilize this action. The best way to prevent
regular users from accessing this action is by making it available in
debug mode only (public servers should never run in debug mode anyway).

* [Connectivity] Split implemenation into multiple files

* [Connectivity] Make web page responsive to user input

* [Connectivity] Make status message sticky

* [Connectivity] Add icon to the status message

* [contents] Add the ability for getContents to return header information

* [Connectivity] Add header information to the reply Json data

* [Connectivity] Add new status (blue) for redirected sites

Also adds titles to status icons (Successful, Redirected, Inactive, Failed)

* [Connectivity] Fix show doesn't work for inactive bridges

* [Connectivity] Fix typo

* [Connectivity] Catch errors in promise chains

* [Connectivity] Allow search by status and update dynamically

* [Connectivity] Add a progress bar

* [Connectivity] Use bridge factory

* [Connectivity] Import Bootstrap v4.3.1 CSS
2019-10-31 22:02:38 +01:00
LogMANOriginal
6bc83310b9 core: Add info button for input fields with title (#1173)
The current solution for titles on input boxes is not obvious to the
user as support varies between bridges. This commit adds an button to
all input boxes with titles in order to make it clear to the user that
additional information is available.
2019-10-31 21:09:44 +01:00
Roliga
c8d5c85c76 formats: Add getMimeType() function (#1299)
Allows getting the expected MIME type of the format's output. A
corresponding MIME_TYPE constant is also defined in FormatAbstract for
the format implementations to overwrite.
2019-10-31 19:00:12 +01:00
somini
d1e4bd7285 [YahtzeeDevDiary] Add new bridge (#1297) 2019-10-31 18:55:08 +01:00
LogMANOriginal
1022b5fdf9 core: Add an option to suppress error reporting (#1179)
Error reporting currently takes place for each error. This can result
in many error messages if a server has connectivity issues (i.e. when
it re-connects to the internet every 24 hours).

This commit adds a new option to the configuration file to define the
number of error reports to suppress before returning an error message
to the user.

Error reports are cached and therefore automatically purged after 24
hours. A successful bridge request does **not** clear the error count
as sporadic issues can be the result of actual problems on the server.

The implementation currently makes no assumption on the type of error,
which means it also suppresses bridge errors in debug mode. The default
value is, however, set to 1 which means all errors are reported.

References #994
2019-10-31 18:49:45 +01:00
LogMANOriginal
e8536ac1b2 core: Add an option to return errors in different formats (#1071)
Bridge errors are currently included as part of the feed to
notify users about erroneous bridges (before that, bridges
silently failed).

This solution, however, can produce a high load of error
messages if servers are down (see #994 for more details).

Admins may also not want to include error messages in feeds
in order to keep those kind of problems away from users or
simply to silently fail by choice.

This commit adds a new configuration section "error" with
one option "output" which can be set to following values:

"feed": To include error messages in the feed (default)
"http": To return a HTTP header for each error
"none": To disable error reporting

Note that errors are always logged to 'error.log' independent
of the settings above.

Closes #1066
2019-10-31 18:40:51 +01:00
Lyra
a0afe36d56 [DownDetectorBridge] Add per-website status fetch. Note that this only fetches the last downtime, as this is the only thing that the API provides. Moreover, the site uses a different ID for every company for every country, resulting in a very large array 2019-10-29 23:14:51 +01:00
Lyra
0b80f9d61c [DownDetectorBridge] Add bridge for DownDetector, and all local variants. Fixes
#1339.
2019-10-29 19:11:28 +01:00
somini
424075981f [EsquerdaNetBridge] Add new Bridge (#1296) 2019-10-29 18:58:12 +01:00
logmanoriginal
c334df91ec composer: Add all details to the composer file
The composer file currently lacks a lot of details, especially the
"name" and "description", but also "require-dev" and "suggest" info.

This commit adds many more details to the composer file and updates
composer.lock for this repository. Technically the project is ready
to be shipped as composer package.
2019-10-28 20:01:19 +01:00
Dominik Thiemermann
f2346fb33e [RevolutBridge] Add new bridge (#1321)
* [RevolutBridge] Add new bridge
2019-10-28 19:49:01 +01:00
Matt DeMoss
8a21fd1476 [BloombergBridge] Remove after site redesign and paywall. (#1238) 2019-10-28 19:27:56 +01:00
somini
2ac44172ac Facebook: Clarify Facebook bridges (#1221)
* Clarify Facebook bridges status

Distinguish between both Facebook bridges by their title.
This preserves all existing URLs.

* Update all URLs to secure HTTPS versions.
* Configure author name abbreviation
* Improve feed names

Use the correct feed name on each bridge.
Make sure the feed names don't repeat the "Facebook" name.
2019-10-28 19:01:04 +01:00
Roliga
4c78721f03 [ParameterValidator] Ensure context has all user provided parameters (#1211)
* [ParameterValidator] Ensure context has all fields

Previously if a bridge had a set of parameters like:

const PARAMETERS = array(
    'ContextA' => array(
        'Param1' => array(
            'name' => 'Param1',
            'required' => true
        )
    ),
    'ContextB' => array(
        'Param1' => array(
            'name' => 'Param1',
            'required' => true
        ),
        'Param2' => array(
            'name' => 'Param2',
            'required' => true
        )
    )
)

and a query specifying both Param1 and Param2 was provided a 'Mixed
context parameters' error would be returned. This change ensures
ContextA in the above example would not be considered a relevant context.
2019-10-28 17:50:55 +01:00
Christian Archer
04be85996d [BastaBridge] Fix PHP 7.4 crash (#1323)
* Inline the function
2019-10-24 21:57:14 +02:00
Joseph
59be6bded2 [GoogleSearchBridge] Replace 'div[id=ires]' with 'div[id=res]' (#1329) 2019-10-16 21:44:41 +02:00
Joseph
46873e14fe [GoogleSearchBridge] Use getURI() to build URLs (#1330)
* [GoogleSearchBridge] Use getURI() to build URLs
2019-10-16 21:44:28 +02:00
Joseph
0f01cc97a4 [StoriesIGBridge] Add timestamp to feed items (#1331) 2019-10-16 21:44:01 +02:00
Joseph
a70e00a76d [SuperbWallpapersBridge] Delete bridge (#1336) 2019-10-16 21:43:38 +02:00
Joseph
f0260c62c3 [StoriesIGBridge] Use getName() to create custom feed titles (#1332)
* [StoriesIGBridge] Use getName()
2019-10-16 21:41:47 +02:00
Roliga
fc5a1526ca [BandcampBridge] Add band and album feeds (#1317)
* [BandcampBridge] Add band and artist feeds

This can return a limited number of the most recent releases by a band,
or a single release/album. Each release may be given a unique article ID
depending on its track list with the "Releases, new one when track track
changes" option, which should make them show up as new articles when
tracks are added or removed. Releases may also be split up to individual
articles for each track with the "Individual tracks" option.

This uses and undocumented API from the Bandcamp Android app. It's much
faster than loading and parsing the website HTML, and seems to fail less
often with more relaxed rate limits. It's still far from perfect in that
regard though.

The "Individual tracks" option generates requests for each individual
track so that can quickly run into rate limits.

The "Individual tracks" option also has a quirk where tracks released
under e.g. a music label will have their artist set to the label instead
of the actual artist of the track. This is a limitation of the API.
2019-10-16 21:37:25 +02:00
Joseph
4c0e234479 [Bridges] Use HTTPS (#1337)
* [Rule34pahealBridge] Use HTTPS
* [KonachanBridge] Use HTTPS
* [Rule34Bridge] Use HTTPS
* [SafebooruBridge] Use HTTPS
* [TbibBridge] Use HTTPS
* [XbooruBridge] Use HTTPS
* [ScmbBridge] Use HTTPS
* [ReporterreBridge] Use HTTPS
* [BastaBridge] Use HTTPS
* [NiceMatinBridge] Use HTTPS
* [ScoopItBridge] Use HTTPS
* [TheCodingLoveBridge] Use HTTPS
* [Shimmie2Bridge] Use HTTPS
* [HDWallpapersBridge] Use HTTPS
* [GiphyBridge] Use HTTPS
* [PickyWallpapersBridge] Use HTTPS
* [ParuVenduImmoBridge] Use HTTPS
* [ElsevierBridge] Use HTTPS
* [CastorusBridge] Use HTTPS
* [CollegeDeFranceBridge] Use HTTPS
* [MangareaderBridge] Use HTTPS
2019-10-16 21:34:28 +02:00
somini
0eab63d728 Update Facebook URL detection (#1334)
* add detectParameters to FacebookBridge.php
2019-10-16 21:32:29 +02:00
floviolleau
b0884e9158 [VieDeMerdeBridge] Add new bridge for quotes from Vie de Merde (#1313)
* Add new bridge for quotes from Vie de Merde
2019-10-03 22:36:08 +02:00
Nicolas Delsaux
b4581418d4 [PlantUMLBridge] Added bridge for PlantUML. Fixes #1191
* Fixes #1191 by implementing the RSS feed of PlantUML releases
2019-10-03 22:30:22 +02:00
sysadminstory
af1566f40d [ZoneTelechargementBridge] URL and name change (#1302)
Annuaire Telechargement has change name again to go back to Annuaire
Telechargement. Fixes #1279
2019-10-03 22:27:10 +02:00
sysadminstory
529e0d0cca [ExtremeDownloadBridge] Update Website URL (#1303)
Website URL was changed.
2019-10-03 22:25:56 +02:00
Paróczai Olivér
a3532804ac [Readme] Small grammar fixes (#1312) 2019-10-03 22:25:05 +02:00
Anchit Bajaj
8c19146d29 [ListverseBridge - add new bridge (#1305) 2019-10-03 22:24:14 +02:00
Anchit Bajaj
2a3d5865ad [FreeCodeCampBridge] - rss feed for FreeCodeCamp (#1311) 2019-10-03 22:23:14 +02:00
Lyra
4d36c9dc30 Merge branch 'master' of github.com:RSS-Bridge/rss-bridge 2019-10-03 22:14:33 +02:00
Lyra
a2e47a88c3 [InstagramBridge] Add option to get direct links 2019-10-03 22:14:21 +02:00
Anchit Bajaj
b09f50853f [ViceBridge] - RSS feed for Vice Publications. (#1310)
* [ViceBridge] - RSS feed for Vice Publications.
2019-10-03 22:02:30 +02:00
lukasklinger
9b5bf565b3 [N26Bridge] Updated bridge to reflect changes on N26 blog (#1295)
N26 made some changes to their blog, this commit fixes the N26Bridge
2019-10-03 21:58:57 +02:00
Lyra
5cc956367f [core] Fix travis 2019-10-03 21:46:49 +02:00
Lyra
548e28249b [ThePirateBayBridge] Remove nested function 2019-10-03 21:46:24 +02:00
Lyra
684c69b0cd [Releases3DSBridge] Remove nested functions 2019-10-03 21:46:09 +02:00
Lyra
3dae4e0801 [JapanExpoBridge] Remove nested function 2019-10-03 21:45:51 +02:00
Lyra
4622d9be1e [ReadComicsBridge] Deleted bridge since website no longer exists 2019-10-03 21:41:22 +02:00
Nicolas Delsaux
76183dcd44 [GQMagazineBridge] Fix article body detection again (Fixes #1280) 2019-10-03 21:26:41 +02:00
Eugene Molotov
50b234d893 [VkBridge] Photo and timestamp fixes (#1287)
* [VkBridge] Correct parsing of photos, fix timestamp for old posts
2019-09-16 21:30:27 +02:00
Eugene Molotov
af48f36fd2 [VkBridge] Switch maintainer (#1288) 2019-09-16 21:29:45 +02:00
Eugene Molotov
7f6ca23e8f [PikabuBridge] Preserve links (#1286)
* [PikabuBridge] Preserve links
2019-09-16 21:28:41 +02:00
oratosquilla-oratoria
1daef22a3d [NFLRUSBridge] Add new bridge (#1285)
* [NFLRUSBridge] Add new bridge
2019-09-16 21:27:01 +02:00
killruana
c694810d9a [MediapartBridge] Fix article parsing
* Only process article item, fix issue #1292
2019-09-16 21:26:19 +02:00
ORelio
f12f6a2dba [DarkReading] Add DarkReading Bridge (#1289) 2019-09-16 21:25:28 +02:00
Lyra
b1be45df6c [Configuration] Bump version to dev.2019-09-12 2019-09-12 17:09:30 +02:00
Lyra
b4f393a5cc [Configuration] Bump version to 2019-07-06 2019-09-12 17:08:15 +02:00
Lyra
29126ebe29 [README] Update list of contributors 2019-09-12 17:07:04 +02:00
triatic
50c971d545 [TwitterBridge] Enable cookies with curl (#1245)
* [TwitterBridge] Enable cookies with curl

Enable cookies in curl, or fall back to `file_get_contents` if in CLI mode with no curl root certificates.
2019-09-12 16:14:48 +02:00
Lyra
7aba7992aa [InstagramBridge] Remove condition that forces cache ignoring 2019-09-11 19:28:46 +02:00
Lyra
48ebed7b38 [InstagramBridge] Fix Instagram stories and user id finding. 2019-09-11 19:08:12 +02:00
Lyra
ccef6b95ad [InstagramBridge] Attempt to fix the queries in order to bypass rate limits 2019-09-10 14:37:50 +02:00
Antoine Turmel
dd5da99a30 [StoriesIGBridge] New bridge (#1187)
* Create StoriesIGBridge.php
2019-09-07 18:43:06 +02:00
Joseph
2ff27b92ff [DailymotionBridge] Use API for playlist and user account feeds (#1217) 2019-09-07 18:42:45 +02:00
Joseph
b47189921f [CuriousCatBridge] Add new bridge (#1216)
* Create CuriousCatBridge.php
2019-09-07 18:37:30 +02:00
floviolleau
f1d3e8c9c9 [AtmoNouvelleAquitaineBridge] Add new bridge for air quality in Bordeaux (#1229)
* Add new bridge for air quality in Bordeaux
2019-09-07 18:36:55 +02:00
triatic
53fbd2a5a0 [FacebookBridge] Prevent sending empty header (#1239)
* [FacebookBridge] Prevent sending empty header

When running in CLI mode, `getEnv('HTTP_ACCEPT_LANGUAGE')` returns `false`. In that case, don't send the `Accept-Language` header.
2019-09-07 18:32:06 +02:00
ORelio
3254a4d7bc [WIRED] Add WIRED Bridge (#1244)
* [WIRED] Add WIRED Bridge
2019-09-07 18:31:19 +02:00
Roliga
52d2d21da5 [TwitchBridge] Add new bridge (#1253)
* [TwitchBridge] Add new bridge
2019-09-07 18:27:44 +02:00
Roliga
abb74f056c [PatreonBridge] Add new bridge (#1254)
* [PatreonBridge] Add new bridge

* [PatreonBridge] Add UID to articles

Patreon changes post URLs when the post title is updated, so set a UID
based on the post ID instead.
2019-09-07 18:26:58 +02:00
dawidsowa
25548b6757 [Rule34pahealBridge] Fix thumbnail uri (#1278) 2019-09-07 18:26:08 +02:00
sysadminstory
cfe433e9e2 [AutoJMBridge] Fix the bridge to follow website changes (#1255)
The Website changed in two way :
- The filter about availability disappeared (and this leads to a
  parameters change, which will break existing bridges, sorry)
- Some HTML change
2019-09-06 10:52:58 +02:00
Nicolas Delsaux
0dfc4ea2c5 [GQMagazineBridge] Adapt to changes, fixes #1280 2019-09-06 10:51:13 +02:00
Lyra
38960df180 [ThePirateBayBridge] Fix PHPCS code violations 2019-09-06 10:55:15 +02:00
Eugene Molotov
b440a6fdc6 [PikabuBridge] Added filtering by user (#1266) 2019-08-28 16:29:49 +02:00
somini
48d0385653 [core] Fix double XML encoding on Atom feed title (#1247) 2019-08-28 16:29:13 +02:00
Roliga
b68c0e0df8 [PirateCommunityBridge] Add new bridge (#1252)
* [PirateCommunityBridge] Add new bridge
2019-08-28 16:28:39 +02:00
Anchit Bajaj
f27b267614 [GuardianBridge] - New bridge for the Guardian (#1249)
* [GuardianBridge] - New bridge for the Guardian
2019-08-28 16:27:45 +02:00
Mitsu
8bff63d9c6 [ThePirateBay] URI fix, add magnet link 2019-08-27 01:18:43 +02:00
Mitsu
2b4a030158 [ThePirateBay] switch back TLD to .org
And the "whack-a-mole" game continues
2019-08-27 00:55:36 +02:00
Rudolf M. Schreier
6a99904e64 [DanbooruBridge] Decode href of HTML element to avoid double escaping. (#1262)
Directly accessing ...->href resulted in a string that contained '&amp;'
instead of '&'. This was later escaped again to '&amp;amp;' in some
formats (e.g. Atom).
2019-08-26 14:26:19 +02:00
sysadminstory
f3c687604f [DealabsBridge] Follow website change (#1256)
A minor website change broke the Bridge. This commit fix it
2019-08-26 14:25:47 +02:00
Lyra
a86a94555d [LeBonCoinBridge] Submit user agent to LBC to get results. 2019-08-26 14:22:58 +02:00
Anchit Bajaj
acc0787b00 [IGNBridge] - New bridge for IGN (#1233)
* [IGNBridge]: New Bridge for IGN
2019-07-31 14:26:43 +02:00
johnnygroovy
c8992650a1 [DavesTrailerPageBridge] Add new bridge (#1246) 2019-07-31 14:17:34 +02:00
Anchit Bajaj
f9f511a849 [NYTBridge] : New bridge for the new york times (#1235) 2019-07-29 12:15:08 +02:00
somini
990719d614 [FabriceBellard]: New Bridge (#1220)
* [FabriceBellard]: New Bridge
2019-07-29 12:12:55 +02:00
triatic
b6be18d585 [contents] Respect passed headers for file_get_contents() (#1234)
* [contents] Respect passed headers for file_get_contents()
2019-07-29 12:05:13 +02:00
Roliga
cf525c964a [WIP][FurAffinityBridge] Add new bridge (#1083)
* [FurAffinityBridge] Add new bridge
2019-07-26 11:02:58 +02:00
Antoine Cadoret
52a4f0860c [LaCentraleBridge] Add new bridge (#1201)
* [LaCentraleBridge] Introduce new bridge
2019-07-26 11:00:55 +02:00
triatic
21b27a1042 [FacebookBridge] Remove relative date from content (#1212)
Remove relative date from content, as well as the separator after it.

As mentioned in #1188.
2019-07-26 10:56:34 +02:00
Léo Maradan
2eee535171 CNET France Bridge (#1214)
CNET France News but with filters on title or url
2019-07-26 10:53:09 +02:00
Anchit Bajaj
da51fc065f [EngadgetBridge] New bridge for Engadget (#1215)
* [EngadgetBridge] New bridge for Engadget
2019-07-26 10:51:20 +02:00
Joseph
e032705c9a [HaveIBeenPwnedBridge] Add item limit parameter, set default limit to 20 (#1219)
* Add `item_limit` parameter to allow user to control number of item returned by bridge. Suggested by @triatic and @somini (code).
2019-07-26 10:47:20 +02:00
Joseph
be27bc9250 Fix malformed URLs (#1222)
Removes 'self::URI' from processUpload() which was creating malformed URLs. Relative URLs are handled by defaultLinkTo() making 'self::URI' unnecessary.
2019-07-26 10:43:18 +02:00
Albirew
75edc1b2b7 [NovelUpdatesBridge] now in https (#1228) 2019-07-26 10:42:41 +02:00
Albirew
c9ea53806d [HentaiHavenBridge] now in https (#1227) 2019-07-26 10:42:19 +02:00
triatic
2bb9480555 [TwitterBridge] Get cookies before sending request (#1232)
* [TwitterBridge] Get cookies before sending request

Twitter now requires cookies to be set before requesting a page. This will fetch the cookies and send them to `getSimpleHTMLDOM()`.

* Formatting fixes
2019-07-26 10:36:59 +02:00
Corentin Garcia
eb942bc498 [UnsplashBridge] Fix bridge (fix issue #965) (#1208) 2019-07-16 16:50:14 +02:00
logmanoriginal
5a0ea423c4 [Configuration] Bump version to dev.2019-07-06 2019-07-06 12:35:36 +02:00
logmanoriginal
2120cc42fb [Configuration] Bump version to 2019-07-06 2019-07-06 12:34:42 +02:00
logmanoriginal
5067501661 [README] Update list of contributors 2019-07-06 12:34:42 +02:00
LogMANOriginal
aea8484ccc [FicbookBridge] Add new bridge (#1185) 2019-07-06 12:29:36 +02:00
logmanoriginal
6b9394dc78 [DemonoidBridge] Remove bridge
The public service demonoid.pw is no longer available and is
currently being rebuild under demonoid.info which hides torrents
behind a login wall. As this is not supported by RSS-Bridge, the
bridge will be removed.

Find more details on Reddit:
https://www.reddit.com/r/Demonoid/
2019-07-06 12:25:23 +02:00
logmanoriginal
4b51d42b8c cache: Keep subfolders in the repository
References #1200
2019-07-06 12:12:59 +02:00
Joseph
d3fbf0d872 Fix bridge description (#1207) 2019-07-06 11:59:55 +02:00
Joseph
41a8eb74a1 [PinterestBridge] Remove search (#1206)
* Remove getSearchResults()
* Remove ''From search' from PARAMETERS array
* Update getURI() and getName()
* Update collectData()
* Add '.rss' to URL in `collectData` instead of in `getURI`
2019-07-06 11:57:48 +02:00
Joseph
7e6c58b67a [HaveIBeenPwnedBridge] Display breach type (#1203)
* Extract breach types for each data breach
* Add paragraph tag
2019-07-06 11:55:31 +02:00
triatic
a31e518a07 [TelegramBridge] Fix forwarded videos (#1202)
Videos forwarded from other channels use a slightly different format, This fixes it.
2019-07-06 11:52:56 +02:00
logmanoriginal
50162f52b6 [XenForoBridge] Fix minor issues with CSS selectors 2019-07-03 19:34:43 +02:00
logmanoriginal
c0edf6e424 [ShanaprojectBridge] Add filter options
- Filter by minimum number of episodes
- Filter by minimum number of total episodes
- Filter by banner image
2019-07-03 19:34:43 +02:00
logmanoriginal
2ea8d73ac1 [ShanaprojectBridge] Return url to current season 2019-07-02 20:46:38 +02:00
logmanoriginal
465cd8c768 [ShanaprojectBridge] Add support for https and cleanup 2019-07-02 20:45:31 +02:00
logmanoriginal
73f4bc078e [CastorusBridge] Fix broken activity selector 2019-06-28 20:31:49 +02:00
Nicolas Delsaux
1add201d3b [WorldOfTanksBridge] Fix bridge (#1197)
* Fix #1196 by better protecting page
2019-06-28 19:32:26 +02:00
Nicolas Delsaux
09113c2594 [GQMagazineBridge] Fix bridge (#1195)
* Fix bridge by changing the way the articles are loaded AND their titles are found
2019-06-28 19:29:32 +02:00
Joseph
c39e642877 [HaveIBeenPwnedBridge] Convert HTML entities to characters (#1198) 2019-06-28 16:08:56 +02:00
Joseph
e2460ead18 [InternetArchiveBridge] Add new bridge (#1186) 2019-06-28 15:45:27 +02:00
logmanoriginal
60c1339612 [InstructablesBridge] Fix after layout changes 2019-06-27 21:05:50 +02:00
logmanoriginal
d324aa5da1 [InstructablesBridge] Update available categories 2019-06-27 20:29:21 +02:00
logmanoriginal
6f24987601 [InstructablesBridge] Fix listCategories() to work with new layout 2019-06-27 20:28:23 +02:00
logmanoriginal
54fb29d443 [InstructablesBridge] Add support for HTTPS 2019-06-27 20:16:53 +02:00
Joseph
ebe463dd08 [TelegramBridge] Set 'username' parameter as required (#1192) 2019-06-27 20:03:18 +02:00
logmanoriginal
987f42d6d4 logo: Add logo to the project
References #1087
2019-06-25 18:42:11 +02:00
logmanoriginal
fa8253c8bf [GiteaBridge] Add new bridge
Gitea is a fork of Gogs and therefore shares most of its features
except for releases.
2019-06-23 09:21:00 +02:00
logmanoriginal
e4444e6432 [GogsBridge] Add new bridge 2019-06-23 09:21:00 +02:00
triatic
3769850ba3 [TelegramBridge] Fix entries for "media too big" (#1184)
When a large video is posted, "Media is too big" appears in web preview. This adds code to detect this and offer a link.
2019-06-23 08:54:52 +02:00
LogMANOriginal
89e3da0b6f [IndeedBridge] Add new bridge (#1166)
Implements a bridge for
https://www.indeed.com/ (or any of the local variants)

Features:
- Takes a company name and returns a list of reviews and comments
- Limit the maximum number of items to return (default: 20)
- No upper limit on the number of items to return
- Search by language code (45 options)
- Supports detectParameters for any supported URL
2019-06-22 18:50:06 +02:00
logmanoriginal
99d4571c6b core: Make RSS-Bridge more usable via mobile devices
Adds styles for display sizes smaller than 768px where
elements are currently hardly usable. Note that RSS-Bridge
is not designed for mobile use, but some users may want
to try things on their mobile phone before using it in
real life applications.

Resolves #796
2019-06-22 18:46:37 +02:00
triatic
69acc6228a [TelegramBridge] Populate author (#1183) 2019-06-22 18:45:15 +02:00
triatic
5e2f0fb626 [TelegramBridge] Prevent double encoding entities (#1182) 2019-06-22 18:44:25 +02:00
triatic
372461b1a3 [TelegramBridge] Fix timestamp for videos (#1181) 2019-06-22 18:34:02 +02:00
logmanoriginal
1591e18027 core: Add context hinting for new feeds
RSS-Bridge currently has to guess the queried context from the data
provided by the user. This, however, can cause issues for bridges
that have multiple contexts with conflicting parameters (i.e. none).

This commit adds context hinting to queries via '&context=<context>'
which can be omitted in which case the context is determined as before.
2019-06-21 19:12:29 +02:00
husimo
e2bca5bb05 [MastodonBridge] Add new bridge (#1178) 2019-06-21 17:30:34 +02:00
logmanoriginal
7926ffad73 [KununuBridge] Improve feed contents
- Add support for ratings
- Add support for benefits
- Fix broken timestamp
2019-06-21 00:00:44 +02:00
logmanoriginal
7ff97c0c7b [HtmlFormat] Dynamically build buttons for other feed formats
Adding or removing feed formats from the "formats/" directory
currently has no effect on the buttons shown in the HTML format.
This can cause errors if users press one of the buttons for a
format that is no longer available on the server.

This commit changes the behavior to dynamically add buttons based
on the available formats. Syndication feeds, however, are no longer
supported as they require knowledge about the content type, which
is not known without further changes to the formats API (may be
added later if there is a demand).

Closes #942
2019-06-19 23:13:37 +02:00
Joseph
1989252608 [TelegramBridge] Add new bridge (#1175) 2019-06-19 22:40:56 +02:00
LogMANOriginal
91e73b00b5 [NationalGeographicBridge] Add new bridge (#1065)
Closes #1029
2019-06-18 22:57:42 +02:00
LogMANOriginal
5c6c79baf4 [VimeoBridge] Add new bridge (#933)
Closes #932
2019-06-18 22:50:31 +02:00
Joseph
99d1343045 [SplCenterBridge] Add new bridge (#1177) 2019-06-18 22:18:52 +02:00
logmanoriginal
14e6dbb645 [ListActionTest] Fix broken test 2019-06-18 19:21:28 +02:00
logmanoriginal
fc8421ed50 format: Refactor format factory to non-static class
The format factory can be based on the abstract factory class if it
wasn't static. This allows for higher abstraction and makes future
extensions possible. Also, not all parts of RSS-Bridge need to work
on the same instance of the factory.

References #1001
2019-06-18 19:15:20 +02:00
logmanoriginal
2460b67886 cache: Refactor cache factory to non-static class
The cache factory can be based on the abstract factory class if it
wasn't static. This allows for higher abstraction and makes future
extensions possible. Also, not all parts of RSS-Bridge need to work
on the same instance of the factory.

References #1001
2019-06-18 19:04:19 +02:00
logmanoriginal
705b9daa0b bridge: Refactor bridge factory to non-static class
The bridge factory can be based on the abstract factory class if it
wasn't static. This allows for higher abstraction and makes future
extensions possible. Also, not all parts of RSS-Bridge need to work
on the same instance of the bridge factory.

References #1001
2019-06-18 18:55:32 +02:00
logmanoriginal
1ada9c26f8 format: Sanitize format name in the format factory
RSS-Bridge currently sanitizes the format name only for the display
action, which can cause problems if other actions depend on formats
as well.

It is therefore better to do sanitization in the factory class for
formats. Additionally, formats should not require a perfect match,
so 'Atom' and 'aToM' make no difference. This will also allow users
to define formats in their own style (i.e. only lowercase via CLI).

References #1001
2019-06-18 18:36:16 +02:00
Corentin Garcia
55e1703741 [EliteDangerousGalnetBridge] Remove duplicate items (#1167) 2019-06-16 20:35:23 +02:00
Tobias Alexander Franke
849eaeb50e [SteamCommunityBridge] Add Workshop category (#1172) 2019-06-16 20:21:48 +02:00
Thibault Couraud
aeca4cfd60 [BAEBridge] Use defaultLinkTo rather than str_replace (#1168) 2019-06-16 19:40:21 +02:00
Thibault Couraud
686f21bc50 [FindACrew] Improve bridge results (#1120) 2019-06-16 19:35:43 +02:00
LogMANOriginal
8dd8be9694 [.gitattributes] Keep files in export for Heroku
Heroku requires the file `app.json` as well as the composer files
`composer.json` and `composer.lock` to deploy a service. Deploy
doesn't work if these files are ignored during export (because of
the way this service deploys projects).

This commit adds comments to .gitattributes to prevent this issue
from re-appearing in the future. All affected lines are commented
out.

Also added some spacing for better readability.

References #1165
2019-06-16 19:15:28 +02:00
logmanoriginal
dfa9c651cd [BridgeList] Change placeholder message in the search bar
The search bar should indicate that searching by URL is
supported.

References #1099
2019-06-13 19:55:10 +02:00
logmanoriginal
6d6d6037a3 [GithubIssueBridge] Don't return error messages in detectParameters()
detectParameters() is called in a loop for all bridges on a URL, thus
if a bridge returns an error message, the output messages get mixed
up and all detect operations fail.

This seems to be a limitation of the detect function for now.
2019-06-13 19:49:54 +02:00
Joseph
2559dbbf49 [BrutBridge] Create custom feed name for each category and edition (#1164) 2019-06-13 19:13:02 +02:00
logmanoriginal
de53120843 [SakugabooruBridge] Remove bridge
The target server for this bridge is no longer reachable and
there doesn't seem to be any attempt to get it back online.
2019-06-12 20:22:53 +02:00
logmanoriginal
b1b7e4edce [DollbooruBridge] Remove bridge
The target site for this bridge has been down for at least a year
now and there doesn't seem to be any attempt to get it back up.
Their twitter account is also silent since 2012, so no harm
removing this bridge.

https://twitter.com/dollbooru?lang=en
2019-06-12 20:11:34 +02:00
logmanoriginal
b27487ace0 [TwitterBridge] Fix detection of retweets on lists
References #1161
2019-06-12 18:27:35 +02:00
logmanoriginal
d005acca83 [TwitterBridge] Add extensive description to keyword search query
References #1163
2019-06-11 21:53:22 +02:00
LogMANOriginal
93de8c239b [README] Remove GooglePlus from supported sites 2019-06-10 15:40:57 +02:00
logmanoriginal
75b0213684 [GithubIssueBridge] Add support for detect action
References #1100
2019-06-10 15:32:57 +02:00
Eugene Molotov
f76a23f0a5 [YoutubeBridge] Add playlist caching (#1162) 2019-06-10 15:31:35 +02:00
logmanoriginal
e4e04a7865 [GithubIssueBridge] Fix broken feed item URLs
References #1100
2019-06-10 00:02:13 +02:00
logmanoriginal
da339fd5cc [GithubIssueBridge] Include issue author comment in the feed
- Add function to build an URL to the GitHub issue comment
- Change scope of internal functions from protected to private
- Use IDs instead of classes as comment selectors, to include the
issue author in the output feed.

References #1100
2019-06-09 20:39:45 +02:00
logmanoriginal
ba116d9ab6 [GithubIssueBridge] Fix bridge after DOM changes 2019-06-09 19:57:48 +02:00
logmanoriginal
ea08445946 [GlassdoorBridge] Fix broken bridge 2019-06-09 19:35:53 +02:00
logmanoriginal
ade09b2aad [XenForoBridge] Fix broken bridge 2019-06-09 19:35:53 +02:00
logmanoriginal
28d46b6721 [ShanaprojectBridge] Fix broken bridge 2019-06-09 19:35:46 +02:00
logmanoriginal
1efb7c7bce [DesoutterBridge] Fix bridge after DOM changes 2019-06-09 19:01:54 +02:00
Joseph
d34411137f [TwitterBridge] Display all images from a tweet (#1160) 2019-06-09 17:24:40 +02:00
logmanoriginal
70542686bb [contents] Fix parsing of incomplete headers
Response headers may contain fields with no values.

Example:
  "Referrer-Policy: "

In this case the current implementation of explode() results in an
error because there is no content after ": ". Changing the delimiter
to ":" and trimming the value manually fixes that issue.
2019-06-09 17:18:08 +02:00
LogMANOriginal
edf10be93a [README] Change color for Guix release to blue
This prevents confusion with the build status for Travis-CI and Docker
2019-06-08 20:36:59 +02:00
LogMANOriginal
a725fdd315 [README] Add logos to badges where applicable 2019-06-08 20:27:41 +02:00
logmanoriginal
84ba0c4a9e [Configuration] Bump version to dev.2019-06-08 2019-06-08 20:12:04 +02:00
logmanoriginal
c17b864242 [Configuration] Bump version to 2019-06-08 2019-06-08 20:04:57 +02:00
logmanoriginal
5ff3d0121c [README] Update list of contributors 2019-06-08 20:04:06 +02:00
Joseph
f00a054e0f [BrutBridge] Add new bridge (#1159) 2019-06-08 19:30:42 +02:00
logmanoriginal
5a9519967b [Exceptions] Add button to search for similar issues on GitHub
Users currently only get one option: to open a new issue on GitHub.
This can, however, result in duplicate issues, which is not desired.

This commit adds a second button to the error message, which links
to the GitHub issues tracker with the search query set to find
errors for the current bridge. That way, users can collaborate
on the same issue.
2019-06-08 17:05:35 +02:00
logmanoriginal
17f587fcbe [index] Don't set the timezone in index.php 2019-06-08 16:16:03 +02:00
logmanoriginal
f28cbecc02 [style] Fix placeholder should be hidden on focus
The placeholder is currently visible on key focus and only hidden
once a user starts typing. This can be confusing and doesn't look
good.

As it turns out, ::placeholder is an official selector:
https://developer.mozilla.org/en-US/docs/Web/CSS/::placeholder

For some reason, listing placeholder selectors with "," doesn't
work on some browsers (tested in FF 60 ESR). Making each of the
selectors explicit works, however.
2019-06-08 15:50:16 +02:00
LogMANOriginal
84450371b5 [README] Remove Deploy to Docker Cloud button
In December 2018 Docker Cloud has become part of Docker Hub:
https://blog.docker.com/2018/12/the-new-docker-hub/

Since then the "Deploy to Docker Cloud" button is broken (error 404)
with no alternative for Docker Hub, so the button should be removed.

Docker images are still available at
https://hub.docker.com/r/rssbridge/rss-bridge/
2019-06-08 15:19:56 +02:00
logmanoriginal
69dd33ac82 [.gitattributes] Use the same indentation style for the entire file 2019-06-08 15:07:08 +02:00
logmanoriginal
95388cdf44 [.gitattributes] Exclude demo bridges from release builds 2019-06-08 15:03:25 +02:00
logmanoriginal
b74dda7af9 [.gitattributes] Exclude Composer and Heroku files from release builds 2019-06-08 15:00:07 +02:00
logmanoriginal
ca1a5feba5 [.gitattributes] Annotate export-ignore sections 2019-06-08 14:58:18 +02:00
Squirrel
69a0498732 [README] Add deploy button to Heroku (#1150)
* Add deploy button to Heroku
* Add composer.json and composer.lock (required by Heroku)
2019-06-08 14:53:26 +02:00
logmanoriginal
3d231a417f bridges: Don't kill scripts with die()
Bridges should generally utilize the API functions instead of killing
the script. Find more information on the Wiki.

- returnServerError
https://github.com/RSS-Bridge/rss-bridge/wiki/The-returnServerError-function

- returnClientError
https://github.com/RSS-Bridge/rss-bridge/wiki/The-returnClientError-function

- returnError
https://github.com/RSS-Bridge/rss-bridge/wiki/The-returnError-function
2019-06-07 20:38:09 +02:00
logmanoriginal
35bd706391 [Configuration] Use common format to report errors to the user
Incorrect configuration values are currently handled individually
for each condition, resulting in a lot of repetitive operations.

This commit adds two new private functions to report errors to the
user and end execution of the script.
2019-06-07 20:27:20 +02:00
logmanoriginal
0e30468e0f [rssbridge] Use PATH_ROOT whenever possible 2019-06-07 19:51:06 +02:00
logmanoriginal
ccf375e917 config: Use global constant for config files
The configuration files are currently hard-coded in the configuration
classes and error messages. However, the implementation should not
rely on specific details like the file name. Instead, the files should
be part of the global definition.

This commit introduces two global constants for the configuration files

- FILE_CONFIG => 'config.ini.php'
- FILE_CONFIG_DEFAULT => 'config.default.ini.php'
2019-06-07 19:48:29 +02:00
logmanoriginal
946a99d334 config: Add [system] => 'timezone'
RSS-Bridge currently statically sets the timezone to UTC which can
result in incorrect timestamps if the server is hosted in another
region.

This commit adds a new configuration parameter to allow admins to
specify their own timezone for their servers. Invalid values will
result in an error message.

Example:

  [system]
  timezone = "UTC"

For compatibility reasons the default value is set to UTC.

This parameter accepts any of the supported timezones listed at
https://www.php.net/manual/en/timezones.php

Closes #956
References #1001
2019-06-07 19:22:51 +02:00
logmanoriginal
e2e0ced055 [Bridge] Improve performance for correctly written whitelist.txt
If the bridge name matches exactly, it is not necessary to perform
a strtolower compare of bridges. In some situations this can lead
to much faster response times (depending on the amount of bridges
in whitelist.txt).
2019-06-06 20:59:33 +02:00
logmanoriginal
d4e867f240 core: Move default bridges to whitelist.default.txt
Default bridges are currently statically defined in index.php, which
is not the right place if we want to keep responsibilities separated.

This commit introduces a new file whitelist.default.txt that holds
the default bridges and which is loaded automatically, if whitelist.txt
doesn't exist.

Due to this it is also no longer necessary to have write permission
for the root directory.

References #1001
2019-06-06 20:53:46 +02:00
Eugene Molotov
b0a780acda [VkBridge] Ignore illegal characters in input html for iconv (#1154) 2019-06-06 20:05:41 +02:00
Antoine Cadoret
1814116d67 [SteamBridge] Follow source changes (#1143)
* Follow source data fetching changes
* Improve media path building
* Improve price fetching and display
2019-06-06 19:59:30 +02:00
LogMANOriginal
d89326fe2d Remove old bridge request template 2019-06-06 19:57:04 +02:00
LogMANOriginal
62198ecfa2 Rename bridge request template
Use the same naming convention for all templates
2019-06-06 19:55:57 +02:00
LogMANOriginal
94e4ef8f27 Add template for generic feature requests 2019-06-06 19:54:34 +02:00
logmanoriginal
6c4098d655 Revert "all: Use ->remove() instead of ->outertext = ''"
This reverts commit 052844f5e1.

There is a bug in ->remove() that causes the parser to incorrectly
identify elements in the DOM tree that shouldn't exist anymore.

References #1151
2019-06-02 13:06:16 +02:00
logmanoriginal
468d8be72d [Exceptions] Fix GitHub query labels for bug reports
All bug reports now use the Bridge-Broken label by default
2019-06-01 22:35:56 +02:00
LogMANOriginal
ed539bacf9 Add issue template for generic bug reports
This commit adds a new template for generic bug reports based on the standard template provided by GitHub.
2019-06-01 22:35:33 +02:00
LogMANOriginal
82a9bb5b1c [.github] Update issue template for bridge requests
* Automatically label bridge requests
* Propose default title for new bridge requests
2019-06-01 22:22:05 +02:00
Eugene Molotov
15c374e317 [PikabuBridge] More options and fixes (#1149)
* Add gif support
* Use page title as feed title
* Implement community support
2019-06-01 21:35:18 +02:00
logmanoriginal
052844f5e1 all: Use ->remove() instead of ->outertext = ''
simplehtmldom 1.9 introduced new functions to recursively remove
nodes from the DOM. This allows removing elements without the need
to re-load the document by using $html->load($html->save()), which
is very inefficient.

Find more information about remove() at
https://simplehtmldom.sourceforge.io/docs/1.9/api/simple_html_dom_node/remove/
2019-06-01 21:29:57 +02:00
logmanoriginal
014b698f67 [html] Use find('*') over custom solution
find('*') wasn't supported in older versions of simplehtmldom but it
is supported now. Thus, all custom implementations can be replaced
by the correct solution.
2019-06-01 21:05:12 +02:00
logmanoriginal
5656792cee [simplehtmldom] Update to version 1.9
Find the release notes at
https://sourceforge.net/projects/simplehtmldom/files/simplehtmldom/1.9/
2019-06-01 20:02:07 +02:00
fulmeek
66c5b732cf [FeedItem] Avoid repeated UID hashing after loading from cache (#1148)
This fixes the following issue:

1. bridge sets unique ids for the items (ids get hashed)
2. items go to the cache
3. on next run items get loaded from cache
4. these items have different ids because they were hashed again
5. they show up twice in feed reader
2019-06-01 19:36:46 +02:00
Joseph
b889e867fd [SoundCloudBridge] Use account avatar as feed icon (#1146) 2019-06-01 15:04:42 +02:00
sysadminstory
b519d350bf [RadioMelodieBridge] Fix bridge after website update (#1145)
- The bridge has been adapted to the new website layout
- The content now shows the header picture below the date
2019-06-01 12:12:17 +02:00
Joseph
2a254855d8 [HaveIBeenPwnedBridge] Add new bridge (#1144) 2019-06-01 12:06:58 +02:00
Nemo
72bcc173eb [Docker] Switch Docker Image to official php base image (#1140)
* Switch Docker Image to official php base image

Switch from the unofficial Alpine+php image to the official php-apache image.
This has 2 advantages:

1. Official image is guaranteed to have regular updates, etc
2. The persistent Docker Alpine DNS Issue goes away;
https://github.com/gliderlabs/docker-alpine/issues/255

* [Docker] Ignore more files from Docker Image
2019-06-01 11:25:01 +02:00
Tobias Alexander Franke
4a60f05fd6 [BinanceBridge] Add new bridge (#1135) 2019-06-01 11:18:30 +02:00
somini
84d48d5614 [QPlayBridge]: New Bridge (#1118)
* [QPlayBridge]: New Bridge
2019-05-29 22:51:52 +02:00
Tobias Alexander Franke
7cf898b5af [SteamCommunityBridge] Add new bridge (#1136)
* [SteamCommunityBridge] Add new bridge
2019-05-29 22:50:04 +02:00
killruana
16bd2aec7a [MediapartBridge] Add new bridge (#1130)
* If no cookie session is defined, use the default rss stream
* Add a parameter for enabling/disabling the single page mode
2019-05-15 21:51:23 +02:00
Dreckiger-Dan
3d87ecbf8c [.gitignore] Add robots.txt to the ignore list (#1128) 2019-05-15 21:40:50 +02:00
Lyra
2cd310c025 Bump version to 2019-05-08 2019-05-08 22:36:22 +02:00
sysadminstory
b764204c3a [YoutubeBridge] Playlist bug fix (#1117)
This commit allow the bridge to parse an infinite number of items of a
Youtube playlist.

It should fix #647 !
2019-05-08 22:17:48 +02:00
Tobias Alexander Franke
a9e2574016 [ArtStationBridge] Added new bridge (#1122)
* [ArtStationBridge] Added new bridge
2019-05-08 22:14:53 +02:00
pofilo
e3f6e1c6db [DELETE] Deletion Google Plus bridge (#1124) 2019-05-08 22:11:50 +02:00
Lyra
8150a73922 [CourrierInternationalBridge] Use newer https-based URL 2019-05-08 22:09:49 +02:00
Lyra
a2f3866383 [RoadAndTrackBridge] Major rewrite, due to the depreciation of their API 2019-05-08 21:57:59 +02:00
Obsidienne
a3446ae77b [AO3Bridge] Add new bridge (#1123)
* [AO3Bridge] Add new bridge
2019-05-06 13:28:42 +02:00
Eugene Molotov
75359bc11b [core] Implemented MemcachedCache (#1000)
* [core] Implemented MemcachedCache
2019-05-03 11:56:07 +02:00
Roliga
fe103974f5 [BadDragonBridge] Add new bridge (#1082)
* [BadDragonBridge] Add new bridge
2019-05-02 22:02:13 +02:00
fulmeek
33c16f8be5 [BakaUpdatesMangaReleasesBridge] Sanitize hash for more solid UIDs (#1113)
This should minimize occasional hiccups on regular updates.
2019-04-30 21:01:48 +02:00
fulmeek
21d3bf3b60 caches: Refactor the API (#1060)
- For consistency, functions should always return null on non-existing data.

- WordPressPluginUpdateBridge appears to have used its own cache instance in the past. Obviously not used anymore.

- Since $key can be anything, the cache implementation must ensure to assign the related data reliably; most commonly by serializing and hashing the key in an appropriate way.

- Even though the default path for storage is perfectly fine, some people may want to use a different location. This is an example how a cache implementation is responsible for its requirements.
2019-04-29 20:12:43 +02:00
sysadminstory
3b8f3da09d [AutoJMBridge] Use title from website for Feed Title (#1093)
* [AutoJMBridge] Use title from website for Feed Title
2019-04-20 22:22:06 +02:00
sysadminstory
f9c4a84c25 [RadioMelodieBridge] Update to support new Website (#1101)
* [RadioMelodieBridge] Update to support new Website
2019-04-20 22:19:22 +02:00
Lorenzo Stanco
7b8dd93a8e [InstagramBridge] Fix image link 2019-04-20 22:15:30 +02:00
somini
8f5151b222 [SIMARBridge]: Add new bridge (#1055)
* [SIMARBridge]: Add new bridge
2019-04-16 09:58:22 +02:00
Lyra
98c2530984 [HDWallpapers] Adapt to some website changes (Fixes #1088). Add wallpapers to enclosures, and select "HD" as the default resolution 2019-04-07 22:02:11 +02:00
sysadminstory
90bf90d167 [BingSearch] Make the bridge compatible with PHP 5.6 (#1084)
* [BingSearch] Make the bridge compatible with PHP 5.6

The use of isset() with an expression is not possible in PHP 5.6. I
fixed it by replacing isset() with "null !== ".
2019-04-07 21:51:48 +02:00
Eugene Molotov
6feda2220e [VkBridge] Add option to hide reposts (#1089) 2019-04-07 21:50:58 +02:00
Lyra
92775abe11 Fix phpcs 2019-04-05 10:59:30 +02:00
Lyra
24cdeabed8 [GithubSearchBridge] Update the bridge to match Github's layout 2019-04-05 10:53:28 +02:00
Roliga
380fdf2e40 [ParameterValidator] Handle missing parameter type (#1057)
* [ParameterValidator] Handle missing parameter type
2019-04-04 22:55:46 +02:00
Tobias Alexander Franke
50c90eb5df [EconomistBridge] Add new bridge (#1067)
* [EconomistBridge] Added new bridge
2019-04-04 22:54:08 +02:00
DJCrashdummy
d9ee9e272e [FDroidBridge] fixed bridge (#1075)
because an additional widget (i guess the language selector) was added to the homepage.
2019-04-04 22:52:59 +02:00
Dreckiger-Dan
4ba0d8bebe Update .gitignore (#1078)
ignore .htaccess .htpasswd
2019-04-04 22:50:33 +02:00
somini
c9b0cd1315 ComboiosDePortugalBridge: HACK: Encode the URL (#1074)
This seems like a weird bug somewhere.

Either the HTML parser should return the valid page, or the CMS should
not convert the URL first, or the URL validation regex is buggy.
2019-04-04 22:48:25 +02:00
Lyra
2dc0c36e9b Merge branch 'master' of github.com:RSS-Bridge/rss-bridge 2019-04-04 22:46:49 +02:00
Lyra
0aa8858551 [RoadAndTrackBridge] Generate a signature key for every client instead of hardcoding it 2019-04-04 22:45:41 +02:00
Thibault Couraud
966d450d27 [FindACrew] Update bridge according new findacrew.net website (#1080)
* update bridge according new crewbay.com website
2019-04-04 22:44:44 +02:00
sysadminstory
291e8c2a23 [AutoJMBridge] Fix bridge after website change (#1081)
* [AutoJMBridge] Fix bridge after website change

The website was totally reworked, so the bridge had to be reworked too.
The bridge parameters changed, therefore old RSS feed will not work
anymore, but it was impossible to do it in another way.
2019-04-04 22:39:39 +02:00
DnAp
b6943de0ca [BingSearch] Add new bridge (#1046) 2019-03-23 16:40:19 +01:00
Aleś Bułojčyk
b9bbc9bdda [FacebookBridge] Fix decoding of cyrillic letters in group names (#842) 2019-03-23 16:39:09 +01:00
logmanoriginal
835e3b1163 [MozillaBugTrackerBridge] Fix typo 2019-03-23 16:30:15 +01:00
Antoine Turmel
3212156925 [MozillaBugTrackerBridge] New Bridge (#916)
This Bridge is a clone of KernelBugTrackerBridge but for Mozilla Bugzilla. There is some difference in the class used to get the right comments.
2019-03-23 16:27:07 +01:00
Dreckiger-Dan
281eaacaeb [HeiseBridge] Add new bridge (#744) 2019-03-23 16:22:44 +01:00
Xurxo Fresco
18d5ef192c [IvooxBridge] Add new bridge (#597) 2019-03-22 21:33:46 +01:00
logmanoriginal
6293c3d33d [FeedItem] Filter duplicate enclosures 2019-03-21 19:42:44 +01:00
logmanoriginal
835af1faf1 travis: Update build script to test more reasonable configurations
PHP nightly recently got updated to dev-8.x, which is not supported
by any of the test scripts. This makes the test pretty useless and
doesn't help in any way.

Instead, the build script should focus on current versions of PHP,
starting from 5.6 to 7.3 (current stable release).

PHP 7.3 is a reasonable version to use for finding breaking changes
in the test scripts (phpunit especially warns about changes). These
tests can fail, of course.
2019-03-20 19:31:34 +01:00
logmanoriginal
88aae6fd95 core: Apply changes to fix broken Travis builds
Travis-CI recently got updated, which causes existing builds to fail.
For example: https://travis-ci.org/RSS-Bridge/rss-bridge/builds/507568117

Indenting multi-line arguments of functions fixes it.
2019-03-20 19:23:22 +01:00
Nemo
684558e276 [StockFilingsBridge] Add new bridge (#1011) 2019-03-17 20:40:21 +01:00
logmanoriginal
d7094b7feb [Configuration] Bump version to dev.2019-03-17 2019-03-17 20:31:17 +01:00
logmanoriginal
ae2c35c18a [Configuration] Bump version to 2019-03-17 2019-03-17 20:28:55 +01:00
logmanoriginal
5b80bcaa04 [README] Update list of contributors
The list acutally didn't change, but it's sorted properly now
(thanks to em92 for the suggestion)
2019-03-17 20:28:15 +01:00
fulmeek
5ea985164e [OneFortuneADayBridge] use date in UTC for seed (#1059) 2019-03-14 19:44:36 +01:00
fulmeek
696afa96d3 [BakaUpdatesMangaReleasesBridge] filter title and groups (#1058)
Baka-Updates Manga uses an asterisk (*) to denote series information have
been updated within the last 24 hours. This is not helpful in a feed.
2019-03-14 19:43:00 +01:00
Roliga
326a707739 [SoundcloudBridge] Update API key (#1062) 2019-03-12 13:29:11 +01:00
LogMANOriginal
1ac66b3fdc [README] Add sqlite3 as requirement for SQLiteCache 2019-03-02 19:42:40 +01:00
logmanoriginal
f450b2e118 [SQLiteCache] Check sqlite3 extension in __construct
Checks if the sqlite3 extension is loaded and throws an error
if it's missing.
2019-03-02 19:33:44 +01:00
sysadminstory
688c950916 [DealabsBridge] Patch unparsable Deal date (#1053)
In case of a unparsable date, the text to DateTime object failed, and
this resulted to a Fatal error while using this DateTime object . To
prvent this fatal error, if the date parsing failse, then a DateTime
object is created with the actual date.
2019-03-02 19:10:57 +01:00
fulmeek
9d85b951f7 [BakaUpdatesMangaReleasesBridge] rework to parse new layout (#1052)
* rework to parse new layout
* skip incomplete rows

The last row could have fewer columns if there are less rows than the items limit. This usually should not happen, though.

* use constant for skipping
2019-03-02 19:09:16 +01:00
somini
dac685b887 [ComboiosDePortugalBridge] Add new bridge (#1049) 2019-03-02 19:05:23 +01:00
ORelio
d37f0c14a0 [LeMondeInformatique] Handle special articles (#1039)
Fix content extraction for special article compiling previous articles
2019-03-02 19:03:29 +01:00
Ryan Liptak
b96c25a3af [BandcampBridge] Update to use newer POST API (#1045)
Bandcamp tags pages have a new layout and now use a POST API endpoint to view each page of releases.

Output of this bridge should be almost the same as before, with a few small improvements:
- Small album image in 'content', larger album image in 'enclosures'
- RSS item titles/authors are appended with the releaser in parentheses if the artist name and the releaser are different (i.e. Record Label's Bandcamp releases an album called Bar by the band named Foo, it would get the title 'Foo - Bar (Record Label)' and the author 'Foo (Record Label)')
2019-02-24 12:08:34 +01:00
fulmeek
dc1b1b13cc [SQLiteCache] Implement cache based on SQLite 3 (#1035) 2019-02-24 12:04:27 +01:00
logmanoriginal
e3588f62bd [Cache] Fix cache types ending on 'cache' are not detected correctly
References #1000
2019-02-24 11:56:43 +01:00
fulmeek
958ba815c7 [OneFortuneADayBridge] Add lucky number feature (#1038) 2019-02-24 11:49:17 +01:00
somini
3d24596a52 [AsahiShimbunAJW] Add new Bridge (#1036) 2019-02-24 11:47:29 +01:00
Lyra
f9ed934c8c Update contributors and bump version 2019-02-19 22:05:06 +01:00
Nono
777c204838 [VMwareSecurityBridge] New Bridge (#1041)
* Create VMwareSecurityBridge.php
2019-02-19 21:53:20 +01:00
Nono
ae40f7b388 [MozillaSecurityBridge] Make the URI unique by adding timestamp (#1005)
* added unique UID + URI 

if UID is mandatory for RSS-Bridge, the unicity of the URI is also mandatory for some reader (like kriss feed).
2019-02-19 21:50:00 +01:00
Lyra
473a62ed44 [RoadAndTrackBridge] Added new bridge 2019-02-12 15:12:04 +01:00
Klimplant
4c58768d4d [CachetBridge] Add new bridge (#1034)
* Fix issue with CachetAPI Pagination

Fixing issue that only the oldest 20 entries were shown.

_Background:_

_Cachet has a, lets call it odd, system of pagination. On the first page you see the incidents first created, so they are not what you want to see. But on the last page you can have 1 or 20 of the newest incidents. So you have to take the incidents from the last page (call it Pmax) and combine them with the incidents from  Pmax - 1._
2019-02-11 21:07:46 +01:00
ORelio
ca9c2abb60 [FeedExpander] Fix item href being used as feed uri (#1033) 2019-02-11 19:07:03 +01:00
logmanoriginal
556a417dd6 core: Add support for custom cache types via config.ini.php
This commit adds support for a new parameter which specifies the type
of cache to use for caching. It is specified in config.ini.php:

 [cache]

 type = "..."

Currently only one type of cache is supported (see /caches). All uses
of 'FileCache' were replaced by this configuration option.

Note: Caching currently depends on files and folders (due to FileCache).
Experience may vary depending on the selected cache type. For now always
check if FileCache is working before testing alternative types.

References #1000
2019-02-06 18:52:44 +01:00
LogMANOriginal
51ee541d5a core: Implement action factory (#1002) 2019-02-06 18:34:51 +01:00
Nova
69cb65c1af [GlowficBridge] Add new bridge (#1031) 2019-02-06 18:20:25 +01:00
David Pedersen
29b187fc12 [AppleMusicBridge] Add new bridge (#1026) 2019-02-06 17:43:20 +01:00
fulmeek
80f6a8b3d4 [MrssFormat] Rework to make it valid RSS 2.0 + Media RSS (#996) 2019-02-06 17:18:33 +01:00
logmanoriginal
32d4da8b76 [Bridge] Fix failed to open stream when reading non-existing whitelist 2019-02-04 17:35:40 +01:00
fulmeek
0063d2c376 [HtmlFormat] minor typographical fix-ups (#1009) 2019-02-04 15:33:13 +01:00
fulmeek
11a39af35c [FormatImplementationTest] Add unit tests for format implementations (#1008) 2019-02-04 14:59:09 +01:00
fulmeek
f65a4076ba [CacheImplementationTest] Add unit tests for cache implementations (#1007) 2019-02-04 14:58:11 +01:00
triatic
25593d9c18 [TwitterBridge] Append username of retweeter to author (#1016)
Append username of retweeter to author. Useful when viewing all unread tweets in an RSS reader which are not sorted within username folders.
2019-02-04 14:56:07 +01:00
LogMANOriginal
394149b114 core: Add item uid (#1017)
'uid' represents the unique id for a feed item. This item is null by
default and can be set to any string value. The provided string value
is always hashed to sha1 to make it the same length in all cases.

References #977, #1005
2019-02-03 20:56:41 +01:00
logmanoriginal
a29512deee [BridgeCard] Don't warn about the 'required' attribute if it is set to false 2019-01-22 19:12:37 +01:00
logmanoriginal
e0db349a57 bridges: Fix bridges that don't pass the unit test 2019-01-22 18:24:32 +01:00
logmanoriginal
d532d0e0c4 [BridgeImplementationTest] Add test for "required" attribute on lists and checkboxes
Lists and checkboxes don't support the "required" flag and should not
define it. Note that the "required" flag can be set to false if so
desired.
2019-01-22 18:22:49 +01:00
logmanoriginal
434c12672f lib: Ignore required attribute on lists an checkboxes
References #1014
2019-01-22 18:11:52 +01:00
fulmeek
ab2e566ee1 [AtomFormat] Update to comply with RFC 4287 (#995)
https://tools.ietf.org/html/rfc4287
2019-01-21 17:22:30 +01:00
fulmeek
493e76e4b9 [BakaUpdatesMangaReleasesBridge] Add new bridge (#999) 2019-01-15 16:36:42 +01:00
logmanoriginal
37d882a8d5 [GlassdoorBridge] Fix incorrect CSS selector 2019-01-13 22:04:21 +01:00
logmanoriginal
bcd7bccc46 vendor: Update PHP Simple HTML DOM Parser to 1.8.1
https://sourceforge.net/projects/simplehtmldom/files/simplehtmldom/1.8.1/

Note: Some bridges may need fixes in their CSS queries if they don't follow
the specification.
2019-01-13 22:02:59 +01:00
logmanoriginal
2def7a04a3 Bump version to dev.2019-01-13 2019-01-13 19:23:59 +01:00
logmanoriginal
3c5b23daa6 [README] Update list of contributors 2019-01-13 19:18:40 +01:00
logmanoriginal
ef6709c402 Bump version to 2019-01-13 2019-01-13 19:15:06 +01:00
Quentin de Longraye
fc96e97d51 [N26Bridge] Add new bridge (#1006)
https://n26.com
2019-01-13 19:12:31 +01:00
fulmeek
600f2290b6 [BridgeImplementationTest] Refactor unit test to check bridges (#980) 2019-01-08 20:02:51 +01:00
triatic
245af35a60 [contents] improve file_get_contents() reporting (#986)
Suppress any errors from file_get_contents() and include the PHP error in the feed instead.
2019-01-06 20:30:02 +01:00
Corentin Garcia
ef4923ae5c [AmazonBridge] Fix parsing of list item (#998)
Closes #993 
Closes #769
2019-01-06 18:38:53 +01:00
Corentin Garcia
18229b5c70 [InstagramBridge] Add author if available in response (#997)
Closes #905
2019-01-06 18:14:23 +01:00
logmanoriginal
3160e62293 [DiscogsBridge] Fix timestamp parsing
References #978
2019-01-05 15:24:44 +01:00
Roliga
f81d1b0846 [TrelloBridge] Fix actions with missing image urls (#987)
When an action is added then removed the image url properties of that
action are missing
2019-01-05 13:27:12 +01:00
fulmeek
8801ac9e64 format: Refactor JsonFormat to JSON Feed version 1 (#988)
JsonFormat now implements https://jsonfeed.org/version/1

Closes #618
2019-01-05 13:20:11 +01:00
fulmeek
288d4de218 bridges: Fix bridges to pass unit test (#984)
* [DealabsBridge] fixed parameters
* [DemonoidBridge] added parameter context names
* [DevToBridge] fixed parameters
* [ExtremeDownloadBridge] fixed parameters
* [GithubIssueBridge] fixed parameters
* [InstagramBridge] added parameter context names
* [MydealsBridge] fixed parameters
* [OnVaSortirBridge] fixed parameters
* [ThingyverseBridge] fixed parameters
* [HotUKDealsBridge] fixed parameters
* [FeedExpanderExample] added proper URI
* [GQMagazineBridge] fixed parameters and getDomain()
* [MozillaSecurityBridge] fixed filename

References #980
2019-01-05 12:29:26 +01:00
Corentin Garcia
f3f33cabed [EliteDangerousGalnetBridge] Add support for others website languages (#992)
* [EliteDangerousGalnetBridge] Add support for others website languages

* [EliteDangerousGalnetBridge] Fix post title
2019-01-03 18:29:29 +01:00
triatic
3e45643418 [index] Fix error when no items defined (#983)
Fix PHP Notice:  Undefined offset: 0. Error below triggers when there are no items:

PHP Notice:  Undefined offset: 0 in C:\php\rss-bridge\index.php on line 249
2018-12-28 16:25:56 +01:00
logmanoriginal
719320e1a4 travis: Fail on deprecation warning
This commit makes Travis fail on deprecation warnings for new versions
of PHP and ensures that checks are made against all versions from PHP
5.6 onwards
2018-12-28 16:15:36 +01:00
triatic
81ee15a161 general: Fix PHP 7.3 deprecation warnings (#982)
Fix PHP 7.3 deprecation warnings. FILTER_VALIDATE_URL implies FILTER_FLAG_SCHEME_REQUIRED and FILTER_FLAG_HOST_REQUIRED since PHP 5.2.1

https://bugs.php.net/bug.php?id=75442
2018-12-28 16:13:03 +01:00
LogMANOriginal
988635dcf3 core: Add FeedItem class (#940)
Add transformation from legacy items to FeedItems, before transforming
items to the desired format. This allows using legacy bridges alongside
bridges that return FeedItems.

As discussed in #940, instead of throwing exceptions on invalid
parameters, add messages to the debug log instead

Add support for strings to setTimestamp(). If the provided timestamp
is a string, automatically try to parse it using strtotime().

This allows bridges to simply use `$item['timestamp'] = $timestamp;`
instead of `$item['timestamp'] = strtotime($timestamp);`

Support simple_html_dom_node as input paramter for setURI

Support simple_html_dom_node as input parameter for setContent
2018-12-26 22:41:32 +01:00
triatic
4095cad9b4 lib: Make cURL module requirement optional (#979)
When running in CLI mode without certificates, do not require curl module to be loaded.
2018-12-26 22:31:30 +01:00
logmanoriginal
e7d3a006c8 global: Fix code violations 2018-12-26 21:58:07 +01:00
logmanoriginal
ce65f51d91 [phpcs] Fix blank line detection
Squiz.WhiteSpace.SuperfluousWhitespace has problems detecting blank
lines in functions when used together with the PSR2 standard.

More information: https://github.com/squizlabs/PHP_CodeSniffer/issues/600

This commit fixes that issue by restoring the original behavior.
It also adds rules for function spacing because the sniff mentioned
above does only work within functions.
2018-12-26 21:39:37 +01:00
Roliga
4b22862295 [DerpibooruBridge] Add new bridge (#949)
New bridge for the derpibooru.org image board.
2018-12-26 21:14:04 +01:00
fulmeek
185a773e74 [DilbertBridge] Fixed URI and item title (#976) 2018-12-26 21:11:45 +01:00
fulmeek
10659dd453 [ModelKarteiBridge] Add new bridge (#975) 2018-12-26 21:10:00 +01:00
fulmeek
6b2a45c1e8 [OneFortuneADayBridge] Add new bridge (#974) 2018-12-26 21:06:16 +01:00
fulmeek
6e4b6fa1cc [OsmAndBlogBridge] Add new bridge (#973) 2018-12-26 20:55:38 +01:00
ORelio
0cad5f24e6 [TheHackerNews] Fix content extraction (#972) 2018-12-26 20:47:02 +01:00
Roliga
cb6ad7c077 [TrelloBridge] Add new bridge (#971)
Adds a new bridge for activity on boards and cards on the trello.com task management site.
2018-12-26 20:44:53 +01:00
Roliga
4438807b26 [SoundcloudBridge] Fix for artists with few tracks (#970)
Artists with less than 10 tracks would return blank articles. This fixes that.
2018-12-26 20:35:05 +01:00
Lorenzo Stanco
6c1d861529 [InstagramBridge] Add link on image and video indication in title (#966)
In item content, the image is now a clickable link to the post;
In item title a ▶ is prepended if the post contains a video; it's impossible to tell from the content image.
2018-12-26 20:32:44 +01:00
triatic
dc83962483 [contents] Use file_get_contents when in CLI mode & no certs (#962)
file_get_contents can natively use system root certificates, so use file_get_contents when in CLI mode with no root certificates for cURL.
2018-12-26 20:04:55 +01:00
logmanoriginal
bb2329fa3a [TwitterBridge] Add option to disable image scaling in feeds
Images in Twitter feeds are currently being scaled by adding ':orig'
(original image) and ':thumb' (thumbnail) to image URIs in the feed.

This can cause issues with feed readers that don't handle colons in
URIs correctly.

Image scaling can now be disabled by adding '&noimgscaling=on' to the
query. This parameter is optional to stay compatible to existing feeds.

References #957
2018-12-12 17:00:12 +01:00
Lorenzo Stanco
758f37b452 [InstagramBridge] Truncate long titles and use full text as content (#961)
- Truncate long titles and use full text as content (using only the first line of text content as title)
2018-12-12 16:44:37 +01:00
logmanoriginal
fb8a064e3a [simplehtmldom] Increase MAX_FILE_SIZE to 10 MB
This fixes an issue where larger pages could not be loaded
because the size limit is too small
2018-12-11 17:16:35 +01:00
logmanoriginal
b00971b2c3 [simplehtmldom] Update parser to version 1.7
- Update parser to version 1.7
https://sourceforge.net/projects/simplehtmldom/files/simplehtmldom/1.7/

References #959

-------------------- CHANGELOG --------------------

- Added code documentation to improve readability
- Added unit tests for `simple_html_dom::$self_closing_tags`
- Added unit tests for `simple_html_dom::$optional_closing_tags`
- Added unit tests for bug reports
  - Added test for bug [#56](https://sourceforge.net/p/simplehtmldom/bugs/56/)
  - Added test for bug [#97](https://sourceforge.net/p/simplehtmldom/bugs/97/)
  - Added test for bug [#116](https://sourceforge.net/p/simplehtmldom/bugs/116/)
  - Added test for bug [#121](https://sourceforge.net/p/simplehtmldom/bugs/127/)
  - Added test for bug [#127](https://sourceforge.net/p/simplehtmldom/bugs/127/)
  - Added test for bug [#154](https://sourceforge.net/p/simplehtmldom/bugs/154/)
  - Added test for bug [#160](https://sourceforge.net/p/simplehtmldom/bugs/160/)
- Added unit tests for memory management of the parser
- Added bit flags to `simple_html_dom::load()`
  - Added bit flag `HDOM_SMARTY_AS_TEXT` to optionally filter Smarty scripts (#154)\
  **Note**: Smarty scripts are no longer filtered by default!\
- Added build script to automate releases
- Added support for attributes without whitespace to separate them
- Improved documentation and readability for `$self_closing_tags`
- Improved documentation and readability for `$block_tags`
- Improved documentation and readability for `$optional_closing_tags`
- Updated list of `simple_html_dom::$self_closing_tags`
  - Removed 'spacer' (obsolete)
  - Added 'area'
  - Added 'col'
  - Added 'meta'
  - Added 'param'
  - Added 'source'
  - Added 'track'
  - Added 'wbr'
- Updated list of `simple_html_dom::$optional_closing_tags`
  - Removed "nobr" (obsolete)
  - Added 'th' as closable element to 'td'
  - Added 'td' as closable element to 'th'
  - Added 'optgroup' with 'optgroup' and 'option' as closable elements
  - Added 'optgroup' as closable element to 'option'
  - Added 'rp' with 'rp' and 'rt' as closable elements
  - Added 'rt' with 'rt' and 'rp' as closable elements
- Clarified meaning of `simple_html_dom->parent`
- Changed default `$offset` for `file_get_html()` from -1 to 0 (#161)
- Changed `simple_html_dom::load()` to remove script tags before replacing newline characters
- `simple_html_dom_node::text()` no longer adds whitespace to top level span elements (only to sub-elements)
- `simple_html_dom_node::text()` adds blank lines between paragraphs
- Normalized line endings in the repository to LF via `.gitattributes`
- Improved performance of `simple_html_dom::parse_charset()` by approximately 25%
- Improved performance of `simple_html_dom::parse()` by approximately 10%
- `str_get_html()` is deprecated and should be replaced by `new simple_html_dom()`
- Removed protected function `simple_html_dom::copy_until_char_escaped()`
- Fixed compatibility issues with PHP 7.3
- Fixed typo (#147)
- Fixed handling of incorrectly escaped text (#160)
- Restore functionality of `$maxLen` in `file_get_html()`
- Fixed load_file breaks if an error ocurred in another script
2018-12-11 17:15:38 +01:00
logmanoriginal
a07ead42a7 Bump version to dev.2018-12-11 2018-12-11 17:07:41 +01:00
logmanoriginal
a11ade3442 Bump version to 2018-12-11 2018-12-11 17:01:16 +01:00
logmanoriginal
3932e7b8ef [README] Update list of contributors
Fix links pointing to the API instead of HTML pages
2018-12-10 22:21:33 +01:00
disk0x
5305c405f6 [SoundcloudBridge] Improve Author, Date, Description (#955)
1. Author Name now doesn't include Episode Title
2. It now fetches Episode Creation Timestamp, to allow correct sorting in podcatchers
3. Description is now the actual show notes, and not an <audio> tag
2018-12-10 21:35:18 +01:00
triatic
1c58c04271 [contents] Better error reporting for cUrl errors (#958)
References #954
2018-12-10 21:20:13 +01:00
logmanoriginal
89218f1da6 [.travis.yml] Fix broken checks
- Remove "sudo:false"
- Update composer installation paths

The Linux infrastructure migration removed support for "sudo:false"

-- https://changelog.travis-ci.com/deprecation-container-based-linux-build-environment-82037
-- https://blog.travis-ci.com/2018-11-19-required-linux-infrastructure-migration
2018-12-07 18:52:37 +01:00
disk0x
30e2b79c38 [SoundcloudBridge] Add RSS enclosures (#952)
Minimum viable code change to get SoundcloudBridge produce feeds that podcatchers like gPodder can understand.
2018-12-04 16:16:19 +01:00
Nono
2184f523cd [MozillaSecurity] New Bridge (#946)
* [MozillaSecurity] New Bridge

Kudo to @teromene & @ArthurHoaro on this one !
2018-11-30 18:25:02 +01:00
triatic
242b6953ed [FB2Bridge] Adapt to Facebook html change (#950) 2018-11-30 18:23:37 +01:00
Roliga
bdcb7a9829 [index] Fix detect action after listBridges rename (#947)
Commit 88b0656 renamed listBridges function which was not taken into
account when adding the detect action.
2018-11-29 16:44:38 +01:00
Pierre Mazière
f4b46e497e [GithubIssueBridge] Be consistent in avoiding is_null
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2018-11-29 16:35:49 +01:00
Pierre Mazière
d5085a4116 [GithubIssueBridge] Fix non existing comments count
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2018-11-29 16:35:45 +01:00
Pierre Mazière
d7cabfca54 [GithubIssueBridge] Fix issue comments and events parsing
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2018-11-29 16:35:41 +01:00
Pierre Mazière
de575982a1 [GithubIssueBridge] Fix most relevant coding style related issues
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2018-11-29 16:35:35 +01:00
LogMANOriginal
3d301fc4ee [contents] Skip caching if the remote server requests no caching (#945)
* Skip caching if Cache-Control defines no-cache
* Skip caching if Cache-Control defines no-store
2018-11-28 17:36:28 +01:00
triatic
263e8872ea core: Don't use server variables in CLI mode (#939) 2018-11-26 18:33:51 +01:00
logmanoriginal
6e9c188a72 [GlassdoorBridge] Fix bridge is marked as executable
References #938
2018-11-26 18:31:25 +01:00
Roliga
49da67cb33 core: Automatically select a bridge based on a URL (#928)
* core: Add bridge parameter auto detection

This adds a new 'detect' action which accepts a URL from which an
appropriate bridge is selected and relevant parameters are extracted.
The user is then automatically redirected to the selected bridge.

For example to get a feed from: https://twitter.com/search?q=%23rss-bridge
we could send a request to:
'/?action=detect&format=Atom&url=twitter.com/search%3Fq%3D%2523rss-bridge'
which would redirect to:
'/?action=display&q=%23rss-bridge&bridge=Twitter&format=Atom'.

This auto detection happens on a per-bridge basis, so a new function
'detectParameters' is added to BridgeInterface which bridges may implement.
It takes a URL for an argument and returns a list of parameters that were
extracted, or null if the URL isn't relevant for the bridge.

* [TwitterBridge] Add parameter auto detection

* [BridgeAbstract] Add generic parameter detection

This adds generic "paramater detection" for bridges that don't have any
parameters defined. If the queried URL matches the URI defined in the
bridge (ignoring https://, www. and trailing /) an emtpy list of parameters is
returned.
2018-11-26 18:05:40 +01:00
sysadminstory
b4dbd191d0 [ZoneTelechargementBridge] Switch to the new Website (#934)
* [ZoneTelechargementBridge] Switch to the new Website

The website zone-telechargement1.org decided that he will be using a new
domain at the end of november :
https://www.annuaire-telechargement.com/

The bridge uses the new domain but still uses the same filename and
class name to keep the existing feed working.
2018-11-20 16:23:17 +01:00
logmanoriginal
e09f452426 [.gitattributes] Exclude files from git archive
Files with the option "export-ignore" are excluded from "git archive"
commands. Release files from GitHub will also ignore those files, so
packages are smaller and don't include unneccessary files.
2018-11-19 18:11:09 +01:00
LogMANOriginal
7b261d1cc2 [contents] Add server side caching for all requests (If-Modified-Since) (#889)
This commit adds a cache for 'getContents' to '/cache/server'. All
contents are cached by default (even in debug mode). If debug mode
is enabled, the cached data is overwritten on each request.

In normal mode RSS-Bridge adds the 'If-Modified-Since' header with
the timestamp from the previously cached data (if available) to the
request.

Find more information on 'If-Modified-Since' here:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Modified-Since

If the server responds with "304 Not Modified", the cached data is
returned.

If the server responds with "200 OK", the received data is written
to the cache (creates a new cache file if it doesn't exist yet).

No changes were made for all other response codes.

Servers that don't support the 'If-Modified-Since' header, will
respond with "200 OK".

For servers that respond with "304 Not Modified", the required band-
width will decrease and RSS-Bridge will responding faster.

Files in the cache are forcefully removed after 24 hours.

Notice: Only few servers actually do support 'If-Modified-Since'.
Thus, most bridges won't be affected by this change.
2018-11-19 17:53:08 +01:00
logmanoriginal
96a518c9e7 [html] Remove todo as it is already implemented 2018-11-18 17:52:45 +01:00
logmanoriginal
0d2ea9a677 [html] Rename parameters for sanitize() 2018-11-18 17:43:34 +01:00
logmanoriginal
66e82e46db [html] Remove todo tags
It is not feasible to use a single 'substr' in the functions
2018-11-18 17:36:00 +01:00
logmanoriginal
54800fcc8d [html] Clarify meaning of strange find() parameter
simple_html_dom currently doesnt support "->find('*')", which is a
known issue: https://sourceforge.net/p/simplehtmldom/bugs/157/

The solution implemented by RSS-Bridge is to find all nodes WITHOUT
a specific attribute. If the attribute is very unlikely to appear
in the DOM, this is essentially returning all nodes.

This is the meaning behind

"->find('*[!b38fd2b1fe7f4747d6b1c1254ccd055e]')"
2018-11-18 17:32:07 +01:00
logmanoriginal
67004556e6 [BridgeCard] Use self:: instead of BridgeCard:: 2018-11-18 16:59:13 +01:00
logmanoriginal
c6a7b9ac64 exception: Remove HttpException class
This class served no particular purpose (other than adding a
layer on top of Exception).
2018-11-18 16:53:21 +01:00
logmanoriginal
dbffbd4d4e [FormatAbstract] Check content type before sending header 2018-11-18 16:30:34 +01:00
logmanoriginal
1c17ffb5c4 [FeedExpander] Add constants for feed types 2018-11-18 16:18:40 +01:00
logmanoriginal
326cfb21cf [FeedExpander] Rename $name to $title 2018-11-18 16:11:38 +01:00
logmanoriginal
8ab1fb86a9 [FeedExpander] Let collectExpandableDatas() return self 2018-11-18 16:03:32 +01:00
logmanoriginal
a9ec3d0d1f [Configuration] Change scope of $config to private 2018-11-18 15:58:34 +01:00
logmanoriginal
ac5bcb62ec [Configuration] Add documentation for defined constants 2018-11-18 15:52:28 +01:00
logmanoriginal
f24ab8b51b [Configuration] Rename $category to $section in getConfig() 2018-11-18 15:45:17 +01:00
logmanoriginal
4348119adf [Configuration] Make file paths explicit 2018-11-18 15:41:43 +01:00
logmanoriginal
fd4124cda2 [Configuration] Make class final
This class is essential to the core library of RSS-Bridge and must
not be extended. This improves predictability for the behaviour of
this class.
2018-11-18 15:34:16 +01:00
logmanoriginal
91f7405297 [Configuration] Throw exception creating objects of this class
This class only provides static functions.
2018-11-18 15:29:50 +01:00
logmanoriginal
85685b7758 [Authentication] Throw exception creating objects from this class
Callers must use Authentication::showPromptIfNeeded()
2018-11-18 15:20:43 +01:00
logmanoriginal
41d02554f3 [YGGTorrentBridge] Add URI to feed items
References #931
2018-11-18 09:41:14 +01:00
logmanoriginal
c4550be812 lib: Add API documentation 2018-11-18 09:41:14 +01:00
Thibault Couraud
b29ba5b973 [CrewbayBridge] Update bridge according to new crewbay.com website (#930) 2018-11-18 09:16:24 +01:00
logmanoriginal
254fe9212a [Debug] Fix debug mode reports indexing error
Error log reports "PHP Notice:  Undefined offset: 2 in /rss-bridge/
lib/Debug.php on line 112" if the array returned by debug_backtrace
does not contain 3 items.

This commit fixes the issue by always using the last element in the
backtrace "end($backtrace)".
2018-11-16 20:19:52 +01:00
triatic
3806895059 [FacebookBridge] Improve titles (#924)
A slightly improved version of #454 and #468 . Build titles from content rather than author + pre-content (which doesn't reflect anything useful).
2018-11-16 15:33:54 +01:00
triatic
599d438a0d [FacebookBridge] Decode all elements in $item (#925) 2018-11-16 15:25:58 +01:00
triatic
e5a6baab96 [TwitterBridge] Decode HTML entities (#926)
Removes duplicate encoding like &amp;quot; (should be &quot;).
2018-11-15 22:00:01 +01:00
logmanoriginal
b47a30ecc1 [rssbridge] Improve documentation 2018-11-15 20:52:17 +01:00
logmanoriginal
860b36c1e3 [Debug] Use self:: instead of Debug:: inside the class 2018-11-15 20:28:26 +01:00
logmanoriginal
3d475572c6 [Debug] Improve documentation 2018-11-15 20:27:32 +01:00
logmanoriginal
59f2d755fe format: Refactor searchInformation
- Rename function to getFormatName
- Add documentation
- Rename variables
- Remove unused variables
2018-11-15 20:16:21 +01:00
logmanoriginal
d7c374bd8c [Format] Add function isFormatName
Returns true if the provided format name is valid
2018-11-15 20:14:43 +01:00
logmanoriginal
6b6ab6486a [Format] Store real path to working directory 2018-11-15 20:06:45 +01:00
logmanoriginal
6c4e239f64 format: Refactor class Format 2018-11-15 20:06:23 +01:00
logmanoriginal
88b0656954 bridge: Rename listBridge to getBridgeNames 2018-11-15 19:43:23 +01:00
logmanoriginal
66b11b8c41 [Bridge] Fix typo 2018-11-15 19:38:14 +01:00
logmanoriginal
1b34d9860e [Cache] Check if class is instantiable 2018-11-15 19:36:01 +01:00
logmanoriginal
6e70d461e1 [Bridge] Add function isBridgeName
This function returns true if the provided name is a valid
bridge name.
2018-11-15 19:33:56 +01:00
logmanoriginal
0a92b5d29b [Bridge] Refactor Bridge::create to improve readability 2018-11-15 19:31:31 +01:00
logmanoriginal
e3849f45ab [Bridge] Use slashes to enclose regex 2018-11-15 19:30:33 +01:00
logmanoriginal
3d9c4a3718 [Bridge] Improve working directory handling
- Initialize with null to prevent leaking configurations
- Check if the working directory is a directory
- Store the real path instead of raw data
- Add final path separator as expected by Bridge::create
2018-11-15 19:28:56 +01:00
logmanoriginal
5f146a257e [Bridge] Change visibility from private to protected 2018-11-15 19:24:43 +01:00
logmanoriginal
936688e08c [Bridge] Fix typos 2018-11-15 19:22:32 +01:00
logmanoriginal
4b5372638c [Bridge] Use self:: instead of Bridge:: inside the class 2018-11-15 19:19:04 +01:00
logmanoriginal
6f4a8f4d03 [Bridge] Rename to in setWorkingDir 2018-11-15 19:17:18 +01:00
logmanoriginal
39652bb050 [Bridge] Rename to 2018-11-15 19:16:37 +01:00
logmanoriginal
fcac5b8b92 [Bridge] Cleanup documentation and exception messages 2018-11-15 19:15:08 +01:00
logmanoriginal
6f7b56cba8 bridge: Rename setDir and getDir to setWorkingDir and getWorkingDir 2018-11-15 19:07:33 +01:00
logmanoriginal
86ac0a4866 [Cache] Fix typos 2018-11-15 19:00:48 +01:00
logmanoriginal
4a99c6e630 cache: Rename setDir and getDir
- Rename setDir to setWorkingDir
- Rename getDir to getWorkingDir
- Rename parameter $workingDir to $dir in getWorkingDir
2018-11-14 20:39:45 +01:00
logmanoriginal
e8442a3bf8 [Cache] Refactor class
general

- Use self:: instead of Cache:: or static::
- Rename $dirCache to $workingDir
- Initialize $workingDir with null

function setDir

- Clear previous working directory before checking input parameters
- Change wording for the exception messages
- Store realpath instead of raw parameter
- Add path separator
  This ensures the path always ends with the path separator, as assumed
  by Cache::create
- Add check if the provided working directory is a valid directory

function getDir

- Use static parameter instead of function variable
- Change wording for the exception message

function create

- Rename parameter $nameCache to $name
- Rename $pathCache to $filePath
- Change wording for the exception messages

function isValidNameCache

- Rename function to isCacheName
- Rename parameter $nameCache to $name
- Explain in the function documentation the meaning of a 'valid name'
- Ensure Boolean return value (preg_match returns integer)
- Check if $name is a string
- Use slashes to enclose the regex
2018-11-14 20:33:44 +01:00
logmanoriginal
427688fd67 [Cache] Add documentation 2018-11-14 17:06:07 +01:00
logmanoriginal
4a6b3654eb [Bridge] Add and rewrite documentation compatible to phpDocumentor
This is the first step in adding documentation to the core library
of RSS-Bridge. The documentation is not yet extracted by phpdoc,
yet may prove useful to anyone interested in starting with RSS-Bridge.
2018-11-13 20:28:17 +01:00
logmanoriginal
5f867c00b4 [CONTRIBUTING] Add new coding style policies 2018-11-13 18:36:37 +01:00
logmanoriginal
c15b25a07d core: Fix PHPCS violations 2018-11-13 18:27:05 +01:00
logmanoriginal
c296e73c18 [phpcs] Add rules for method declarations in classes 2018-11-13 18:25:37 +01:00
logmanoriginal
007ee4d858 [Bridge] Fix broken bridge initialization
Commit e26d61e introduced a bug that causes the error message "The
bridge you [sic!] looking for does not exist." if the bridge name
specified in the query ends on "Bridge"
(i.e. '&bridge=SoundcloudBridge'), while other queries work fine
(i.e. '&bridge=Soundcloud').

This commit fixes that issue by sanitizing the bridge name before
creating the class.

References #922
2018-11-13 17:36:06 +01:00
Thomas Dalichow
dd95ec6200 core: Fix grammar (#923) 2018-11-13 17:24:36 +01:00
Eugene Molotov
d951000c23 [index] Redirect _cache_timeout requests if the option is disabled (#894)
Requesting `_cache_timeout` on servers where this option is disabled currently results in the error message 'This server doesn\'t support "_cache_timeout"!'. This commit changes that behavior to redirect to the query without `_cache_timeout`.
2018-11-13 17:19:00 +01:00
triatic
51634a72e0 [TwitterBridge] Reorder quoted tweets (#921)
Put content before quoted tweet to match the display order on Twitter
2018-11-12 19:59:46 +01:00
logmanoriginal
78c69b08f0 [index] Fix invalid bridge name FlickrExploreBridge => FlickrBridge 2018-11-10 22:33:19 +01:00
logmanoriginal
3bb3353897 [Bridge] Use static variable in listBridges()
This prevents the function from re-loading the same data over and over
again. Instead the same data is returned on each call, during a single
request.
2018-11-10 22:31:40 +01:00
logmanoriginal
e26d61ec0a core: Refactor bridge whitelisting
- Move all whitelisting functionality inside Bridge.php
- Set default whitelist once in index.php using Bridge::setWhitelist()
- Include bridge sanitizing inside Bridge.php
    Bridge::sanitizeBridgeName($name)

Bridge.php now maintains the whitelist internally.
2018-11-10 22:26:58 +01:00
logmanoriginal
a0490e3673 core: Add Debug::isEnabled() and Debug::isSecure()
Also adds documentation to Debug.php!

* Debug::isEnabled()

Checks if the DEBUG file exists on disk on the first call (stored in
memory for the duration of the instance). Returns true if debug mode
is enabled for the client.

This function also sets the internal flag for Debug::isSecure()!

* Debug::isSecure()

Returns true if debuging is enabled for specific IP addresses, false
otherwise. This is checked on the first call of Debug::isEnabled().
If you call this function before Debug::isEnabled(), the default value
is false.
2018-11-10 20:50:34 +01:00
logmanoriginal
c63af2e7ad core: Add separate Debug class
Replaces 'debugMessage' by specialized debug function 'Debug::log'.
This function takes the same arguments as the previous 'debugMessage'.

A separate Debug class allows for further optimization and separation
of concern.
2018-11-10 20:03:05 +01:00
logmanoriginal
9379854f7a core: Define path to whitelist.txt in rssbridge.php 2018-11-10 19:51:37 +01:00
logmanoriginal
ecdac1b089 core: Add path separator to PATH_CACHE 2018-11-10 19:48:05 +01:00
logmanoriginal
2104fc4d58 core: Move initialization for static paths to rssbridge.php
Bridge, Format and Cache are all part of the core logic of RSS-Bridge
and should therefore be initialized centrally
2018-11-10 19:42:54 +01:00
logmanoriginal
697d63bb96 core: Rename RssBridge.php to rssbridge.php
Using lower case letters because the file doesn't implement a class.
2018-11-10 19:01:57 +01:00
logmanoriginal
2bb13169b4 [Configuration] Use FILTER_VALIDATE_EMAIL on admin/email
This prevents including arbitrary data as email address.
2018-11-10 18:43:16 +01:00
logmanoriginal
4713fb6190 Bump version to dev.2018-11-10 2018-11-10 18:11:49 +01:00
logmanoriginal
a08811f147 Bump version to 2018-11-10 2018-11-10 18:04:58 +01:00
logmanoriginal
a935e310ff travis: Rewrite checks and add PHP compatibility tests
- Remove HHVM

HHVM recently announced ending PHP support:
https://hhvm.com/blog/2018/09/12/end-of-php-support-future-of-hack.html

"HHVM v3.30 will be the last release series where HHVM aims to support
PHP. [...] Ultimately, we recommend that projects either migrate
entirely to the Hack language, or entirely to PHP7 and the PHP runtime."

RSS-Bridge never "officially" supported HHVM, so support can be removed.

- Use composer for all versions

PHP 5.6 is using PEAR, while all other versions use Composer to manage
packages and dependencies. This commit removes PEAR for PHP 5.6 in favor
of Composer. This also simplifies the script.

- Add PHP compatibility tests

Uses https://github.com/PHPCompatibility/PHPCompatibility

RSS-Bridge supports PHP 5.6 or higher. This commit adds tests to check
compatibility and detect breaking changes.

"phpcompatibility.xml" contains the ruleset.

Notice: Technically RSS-Bridge requires PHP 5.6.1, but for some reason
PHPCompatibility doesn't accept "5.6.1" for "testVersion". This is why
INI_SCANNER_TYPED is excluded from tests.

- Rearrange tests

PHP 5.6:
  - Coding style (phpcs.xml)
  - PHP compatibility (phpcompatibility.xml)

PHP 7.0:
  - Coding style (phpcs.xml)
  - Unit tests (phpunit.xml) - using stable release of PHPUnit

PHP nightly:
  - Coding style (phpcs.xml)
  - Unit tests (phpunit.xml) - using latest version of PHPUnit
  - PHP compatibility - no exceptions for PHP 5.6+

- Documentation added to improve maintainability
2018-11-09 20:49:47 +01:00
LogMANOriginal
7e3787a185 .github: Add issue template for bridge requests
This commit adds an issue template for bridge requests, automatically suggested to anyone reporting a new issue.

References https://gist.github.com/4c38d575de8f1edd386fe7c2d529ab6f

Closes #759
2018-11-08 19:45:16 +01:00
logmanoriginal
039c032798 Add folder for GitHub related files
For more information see
https://help.github.com/articles/setting-guidelines-for-repository-contributors/
2018-11-08 19:31:33 +01:00
logmanoriginal
cb91d9cce8 [FacebookBridge] Fix media origin info is not inside a tag
References #912
2018-11-08 19:24:14 +01:00
triatic
bf91f106b4 [FacebookBridge] Remove "Posts" from author name (#917) 2018-11-08 19:04:58 +01:00
logmanoriginal
0b2ede35cd [FacebookBridge] Don't remove origin information from embedded media
References #912
2018-11-08 18:59:12 +01:00
logmanoriginal
5842bdfc83 [FacebookBridge] Simplify implementation 2018-11-08 18:45:25 +01:00
logmanoriginal
68ee24d6bd [FacebookBridge] Remove videos and views
This commit adds filters to remove embedded videos and view counts from
all posts. This doesn't remove the preview image for videos, which are
embedded separately.
2018-11-08 18:36:11 +01:00
logmanoriginal
104ae2298e [FacebookBridge] Remove hidden elements
Hidden elements are used for error conditions and generally made
visible using JavaScript. Since RSS-Bridge doesn't support JS, these
error messages are shown in the final feed. For example:

"It looks like you may be having problems playing this video. If so,
please try restarting your browser."

This commit removes all hidden elements to prevent error messages being
added to the feed.

- "It looks like you may be having problems playing this video. If so,
please try restarting your browser."
2018-11-08 18:24:05 +01:00
logmanoriginal
7026684e34 [FacebookBridge] Don't remove description of embedded media
FB includes origin information (i.e. "YOUTUBE.COM") as well as
descriptions with embedded media (images and video).

These details are currently being removed by the bridge.

This commit changes implementation to only remove origin information
and keep the media description in place. The media description consists
of two elements - title and description. The title provided by FB is
included in an anchor, which gets replaced by a paragraph with the
same contents to improve readability.

References #912
2018-11-08 18:12:57 +01:00
teromene
0b792d77eb [Rue89Bridge] Fix style. 2018-11-07 23:16:28 +01:00
teromene
110b865a54 [Rue89Bridge] Entirely rewrite the bridge. It now uses the JSON api. 2018-11-07 23:13:45 +01:00
teromene
19a7f10160 [InstagramBridge] Support Instagram Locations. Fixes #705. 2018-11-07 22:17:53 +01:00
Antoine Turmel
42e25e7fc0 [OnVaSortirBridge] New Bridge (#914)
Bridge to expand OnVaSortir RSS feed to get the full description of an event
2018-11-07 18:52:29 +01:00
logmanoriginal
4b7fea5ebc [RssBridge] Include interfaces once 2018-11-06 19:23:32 +01:00
logmanoriginal
95bd206e9d core: Move REPOSITORY from index.php to RssBridge.php 2018-11-06 18:53:35 +01:00
logmanoriginal
9910310652 [BridgeImplementationTest] Use PATH_LIB_BRIDGES 2018-11-06 18:46:18 +01:00
logmanoriginal
12f0e5a360 [RssBridge] Include path separator in PATH_* 2018-11-06 18:44:45 +01:00
logmanoriginal
81ba96ff94 core: Add PATH_LIB_BRIDGES, PATH_LIB_FORMATS and PATH_LIB_CACHES
- PATH_LIB_BRIDGES defines the path to bridges
- PATH_LIB_FORMATS defines the path to formats
- PATH_LIB_CACHES defines the path to caches

Include constants in RssBridge.php for consistency
2018-11-06 18:42:27 +01:00
logmanoriginal
984f0b24d0 [RssBridge] Rename PATH_VENDOR to PATH_LIB_VENDOR
This improves clarity for the parameters
2018-11-06 18:39:05 +01:00
logmanoriginal
2126db84ac core: Replace CACHE_DIR by PATH_CACHE
Move CACHE_DIR from index.php to /lib/RssBridge.php and change name
to PATH_CACHE.

PATH_CACHE is one of the core paths of RSS-Bridge and should therefore
be defined in the core file RssBridge.php.
2018-11-06 18:35:43 +01:00
logmanoriginal
4bf45df18e [RssBridge] Simplify documentation for this file
- Remove file documentation and license remark (defined in repository
scope - see README / UNLICENSE)

- Remove example usage (if necessary should be included in the Wiki)
2018-11-06 18:31:48 +01:00
logmanoriginal
a88b148d20 [RssBridge] Add PATH_LIB
Add constant PATH_LIB, pointing to '/lib' to make the include process
same for vendor and lib files.
2018-11-06 18:24:07 +01:00
logmanoriginal
f564925ba0 [RssBridge] Use require_once instead of require
"The require_once statement is identical to require except PHP will
check if the file has already been included, and if so, not include
(require) it again."

-- http://php.net/manual/en/function.require-once.php
2018-11-06 18:15:10 +01:00
logmanoriginal
22e8f8b4aa [RssBridge] Skip searching vendor files
Vendor files (simple_html_dom.php and urljoin.php) are included in the
repository and therefore shipped with all releases. If one of the files
is missing, either the repository or the release is incomplete.

PHP will generate error messages if either of the files is missing, so
there is no need to check availability manually unless it is done for
all files (which doesn't make sense because they are part of the
repository).
2018-11-06 18:11:18 +01:00
logmanoriginal
bfae04d1fe [RssBridge] Include __DIR__ in PATH_VENDOR 2018-11-06 18:08:53 +01:00
teromene
723bd1150a Remove tracking codes from Facebook posts 2018-11-06 16:58:58 +01:00
Thibault Couraud
53d2fbe3a5 [FindACrewBridge] Implement bridge for findacrew.net (#901)
* [FindACrewBridge] Implement bridge for findacrew.net - sailing boats offers
2018-11-06 14:57:54 +01:00
Thibault Couraud
3babd02658 [CrewbayBridge] Implement bridge for crewbay.com (#902)
* [CrewbayBridge] Implement bridge for crewbay.com - sailing boats offers
2018-11-06 14:56:23 +01:00
logmanoriginal
3031fa406d core: Set code in header() instead of calling http_response_code() 2018-11-05 19:29:01 +01:00
logmanoriginal
85c34a0960 [CHANGELOG.md] Remove file
The latest changelog is available at
https://github.com/RSS-Bridge/rss-bridge/releases
2018-11-05 19:14:44 +01:00
logmanoriginal
5deb86acff core: Replace PHP_VERSION_REQUIRED by static text
The required PHP version is used in one place only and
therefore shouldn't require a constant
2018-11-05 19:07:33 +01:00
logmanoriginal
946e66e9df core: Use REPOSITORY constant where applicable 2018-11-05 19:05:59 +01:00
logmanoriginal
1a00dfa412 [index.php] Change user agent to constant and include current version 2018-11-05 19:04:30 +01:00
Corentin Garcia
0f8443e1d3 [RainbowSixSiegeBridge] Fix missing news (#908) 2018-11-05 18:20:17 +01:00
Albirew
7d474e5361 [ThePirateBayBridge] Fix TLD from .org to .wf (#907) 2018-11-05 18:17:46 +01:00
Corentin Garcia
8c97953211 [CommonDreamBridge] Promote to secure bridge (fix #777) (#909) 2018-11-05 17:32:11 +01:00
logmanoriginal
d987ceec73 [CONTRIBUTING.md] Include all policies and link to the Wiki 2018-11-05 14:07:14 +01:00
logmanoriginal
392e3ff6c7 phpcs: Fix violations 2018-11-05 12:55:58 +01:00
logmanoriginal
e295dc5a79 [phpcs] Add check for concatenation operator spacing
The concatenation operator should have one space before and after
2018-11-05 12:52:18 +01:00
logmanoriginal
b9f6bc8197 [XenForoBridge] Fix broken conditions
Restore functionality for https://xenforo.com/community/
2018-11-05 12:19:45 +01:00
logmanoriginal
9c1c0f2974 [XenForoBridge] Fix broken checks 2018-11-05 12:05:14 +01:00
logmanoriginal
65da157fff [XenForoBridge] Add new bridge
Adds a bridge for forums powered by XenForo (see https://xenforo.com).

Support between forums may vary due to ever changing versions with no
clear distinction. Especially timestamps may not work depending on the
supported language (should currently work on en-US and de-DE).

Tested on

- https://xenforo.com/community/
- http://www.ign.com/boards/

Notice: XenForo provides RSS feeds for forums (but not specific topics).
For example: https://xenforo.com/community/forums/-/index.rss
2018-11-05 12:00:12 +01:00
triatic
5fe943562a [FB2Bridge] Prevent shared post duplication (#904)
Prevent shared posts appearing twice in feed.
2018-11-05 11:46:56 +01:00
Thibault Couraud
c58331f74d [BAEBridge] Add bridge for bourse-aux-equipiers.com (#903) 2018-11-05 11:38:22 +01:00
Antoine Turmel
145a46ae1d [ThingiverseBridge] Add new bridge (#869) 2018-11-05 11:27:32 +01:00
mr-flibble
1a7a7bad98 [contents.php] Fix typo (#900)
This fixes "The requested resouce cannot be found!" on line 67
2018-11-05 11:10:32 +01:00
Yardena Cohen
27d6a22675 core: Display optional administrator email (#896) 2018-11-05 10:46:44 +01:00
teromene
b55ec51e0e Fix timestamp decoding 2018-11-04 21:50:18 +01:00
hunhejj
07b4c72d5d [InstagramBridge] Don't add duplicated urls when parsing Instagram stories (#715) 2018-11-03 12:12:37 +01:00
logmanoriginal
2e6cbd1ce7 [GitHubGistBridge] Fix broken bridge
`defaultLinkTo` makes anchors point to the correct path which broke
parsing because it expected href to start with `#gistcomment`.

This commit changes the implementation to make `defaultLinkTo` point
to the correct page (using `getURI` instead of `self::URI`) and search
with `*=` instead of `^=`.
2018-11-03 11:56:51 +01:00
LogMANOriginal
2ac2f3dc66 [README] Add info about feed readers
References #892
2018-11-02 11:45:45 +01:00
logmanoriginal
e2dfea2b77 [index.php] Filter parameter '_error_time' from queries
The parameter is used in error feeds. Since RSS-Bridge returns valid
feeds for error conditions, feed readers may attempt to access the
URI returned for the feed item in order to collect additional data,
thus including the parameter '_error_time' in the query.

This results in another error message, because it is an invalid input
parameter. Filtering the parameter allows RSS-Bridge to return the
original feed.

References #882
2018-11-02 11:05:48 +01:00
Yardena Cohen
c4896c7791 [Configuration] Fix open_basedir warnings (#887)
If .git/HEAD isn't in open_basedir it'd throw ugly warnings.
Suppress errors while checking if file is readable
2018-10-27 10:53:45 +02:00
logmanoriginal
7621784598 bridges: Add favicon to bridges missing it
Adds favicon to bridges that support it. Some sites prevent downloading
favicons, those bridges are left untouched.

Affected bridges:

- AutoJMBridge
- BandcampBridge
- BlaguesDeMerdeBridge
- BloombergBridge
- BundesbankBridge
- ChristianDailyReporterBridge
- ContainerLinuxReleasesBridge
- DailymotionBridge
- DiceBridge
- DribbbleBridge
- EliteDangerousGalnetBridge
- ElsevierBridge
- FacebookBridge
- FB2Bridge
- FDroidBridge
- FierPandaBridge
- GooglePlusPostBridge
- JapanExpoBridge
- KATBridge
- KernelBugTrackerBridge
- LegifranceJOBridge
- NotAlwaysBridge
- NyaaTorrentsBridge
- PinterestBridge
- RadioMelodieBridge
- RainbowSixSiegeBridge
- SupInfoBridge
- TagBoardBridge
- TebeoBridge
- TheTVDBBridge
- WhydBridge
- ZoneTelechargementBridge
2018-10-26 19:10:58 +02:00
logmanoriginal
1cfe939927 [AskfmBridge] Fix broken bridge
References #774
2018-10-24 18:33:07 +02:00
logmanoriginal
c56f7abc2a [FacebookBridge] Reduce occurrence of HTTP error 302
Facebook returns "HTTP/1.1 302 Found" when requesting:
  https://www.facebook.com//pg/username/posts?_fb_noscript=1
Automatically redirecting to:
  https://www.facebook.com/username/posts/

We receive a positive response faster when directly requesting the
correct page:
  https://www.facebook.com/username/posts?_fb_noscript=1

Notice: This is just a minor adjustment to improve performance while
requesting data from the server. The previous version worked fine as
well.
2018-10-24 17:27:46 +02:00
logmanoriginal
e3030cbbfd [InstagramBridge] Reduce occurrence of HTTP error 301
Instagram returns "HTTP/1.1 301 Moved Permanently" on each request
to "https://instagram.com/" because the correct location is
"https://www.instagram.com/".

Instagram will respond with "HTTP/1.1 301 Moved Permanently" if the
URI for the requested user doesn't end with a slash.

Notice: This is only a minor enhancement to prevent error 301 from
happening. The previous version worked fine as is.
2018-10-24 16:42:28 +02:00
logmanoriginal
953c6e1022 [contents] Skip setting options on empty array 2018-10-24 16:28:26 +02:00
logmanoriginal
dbd44f64dd [contents] Add debug messages for 'getContents'
Adds additional messages to the error log when fetching contents. The
data is helpful in finding issues with receiving contents from servers.

References: #879, #882, #884
2018-10-24 16:10:33 +02:00
logmanoriginal
89ca42da54 [index] Always write exceptions to error.log
Exceptions are reported to users, but they do not necessarily appear
in the error log on the server. Using 'error_log' we can explicitly
write exceptions and error messages to the log file, using the
standard PHP message format.

For more information see https://stackoverflow.com/a/26867035
2018-10-24 15:58:12 +02:00
sysadminstory
b4b5340b7e [ZoneTelechargementBridge] Make the bridge more robust to URL change (#881)
Using the classical www.zone-telechargement1.org as base URL, the bridge will
always be redirected to the actual wwX.zone-telechargement1.org final URL. This
makes the bridge more robust to URL changes.
2018-10-22 19:22:02 +02:00
Eugene Molotov
a508dddb36 [core] Fixed broken caching (#880) 2018-10-22 19:14:49 +02:00
logmanoriginal
cb488d9d8c [FacebookBridge] Fix broken feeds
This commit collects the original contents from a different
tag to prevent this issue. The root cause is unknown but closely
related to the regex.

References #877
2018-10-20 15:45:20 +02:00
Antoine Turmel
9820ad5c0f [BridgeCard] Fix checkbox default value (#874)
The current solution just output "1" when checked instead of "checked"
2018-10-20 13:14:46 +02:00
Antoine Turmel
ea2d54523d [EtsyBridge] Fix bridge and correct typos (#873) 2018-10-20 13:08:03 +02:00
Eugene Molotov
87d218296e [YoutubeBridge] Fix playlist mode (#876)
* Corrected duration text selector
* Request YouTube page with English localization
* Filter video items in the beginning of the loop
2018-10-20 12:43:48 +02:00
teromene
afd5ef0f1d [FB2Bridge] Add images support
[FB2Bridge] Add basic "cards" support
2018-10-18 21:10:02 +02:00
teromene
30bc5179c2 Fix number of fetched items.
Strip the username.
2018-10-18 18:44:11 +02:00
teromene
7596be65f2 Use a new URL for the cursor. Should fix #851.
Remove the "...More" item in the output
Remove the information card data
2018-10-18 18:07:07 +02:00
Eugene Molotov
16f0ee7104 [InstagramBridge] added caption existance check in getInstagramStory (#865)
* [InstagramBridge] added caption existance check in getInstagramStory

* [InstagramBridge] Coding policy fixes
2018-10-18 16:45:03 +02:00
fluffy
e0323f06cd update php-urljoin (#867) 2018-10-18 16:43:39 +02:00
logmanoriginal
717b0bdd9c Fix items link to localhost
References #864
2018-10-16 19:16:51 +02:00
logmanoriginal
62d737efe2 Replace emoticon images by their textual representation
References #850
2018-10-16 19:02:55 +02:00
triatic
6fce03daa7 [FB2Bridge] Add updated timestamps to each post (#849)
Additionally, exclude shared posts from output since they already exist inside other posts.
2018-10-16 18:34:39 +02:00
logmanoriginal
7561c0685d [FacebookBridge] Fix 'SpSonSsoSredS' text in title
The function 'defaultLinkTo' applied to the source HTML does break
regex matches later in the bridge. We need to apply the function
right before adding the contents to the item for the bridge to work
properly.

References #856
2018-10-15 19:53:46 +02:00
logmanoriginal
f48eac854f Bump version to 'dev.2018-10-15' 2018-10-15 18:59:03 +02:00
logmanoriginal
a87e7781b1 Bump version to 2018-10-15 2018-10-15 18:54:53 +02:00
logmanoriginal
0dc761d6cf [README] Update authors
Not sure why, but the GitHub API responded with false results the
last time. Cleaning up to reflect current list of contributors.
2018-10-15 18:53:27 +02:00
logmanoriginal
d14f8e3c83 [BundesbankBridge] Add new bridge 2018-10-15 18:38:42 +02:00
logmanoriginal
b4aea21f71 [DesoutterBridge] Add new bridge 2018-10-15 18:35:49 +02:00
logmanoriginal
c06a09fe99 [GlassdoorBridge] Add new bridge 2018-10-15 18:33:02 +02:00
sysadminstory
704ad50607 [DealabsBridge] Follow website changes (#852)
Pepper changed the CSS class of some elements. The bridge was changed to
follow these changes.
2018-10-15 18:25:04 +02:00
sysadminstory
d89c65d219 [ZoneTelechargementBridge] Update the base URL and make URI unique (#853)
- Base URL updated
- Show name has different styles on the Website, use another way to get the show name
- Entry URIs are now unique to make sure RSS readers don't treat episodes as duplicates
- No more new lines in the feed or item title
2018-10-15 18:23:08 +02:00
sysadminstory
9a3c776096 [ExtremeDownloadBridge] Make URI and titles unique (#854)
- Entry URIs are unique to make sure RSS readers don't treat episodes as duplicates
- Titles are unique to make sure RSS readers don't treat streams and downloads as duplicates
2018-10-15 18:19:57 +02:00
triatic
85e8a67568 [MrssFormat.php] Prevent PHP Notice (#858)
Prevent PHP Notice when running in CLI mode
2018-10-15 18:14:06 +02:00
Nicolas Delsaux
ee158468fa Expanded Sexactu to cover the whole GQ magazine (#861)
The bridge has been expanded to better cover the whole GQ magazine.
It should support all countries (provided they all use the same absurdly shitty publication system).
It is guaranteed to be only tested with sexactu articles (that I now obtain by loading Maïa Mazaurette author page).
2018-10-15 18:09:20 +02:00
logmanoriginal
5779f641c0 [FacebookBridge] Add option to limit number of returned items
This commit adds a new optional parameter 'limit' which can be used
to limit the number of items returned by this bridge (i.e. '&limit=10')

As requested in #669
2018-10-15 17:35:10 +02:00
LogMANOriginal
b90bcee1fc Return exceptions in requested feed formats (#841)
* [Exceptions] Don't return header for bridge exceptions
* [Exceptions] Add link to list in exception message

This is an alternative when the button is not rendered
for some reason.

* [index] Don't return bridge exception for formats
* [index] Return feed item for bridge exceptions
* [BridgeAbstract] Rename 'getCacheTime' to 'getModifiedTime'
* [BridgeAbstract] Move caching to index.php to separate concerns

index.php needs more control over caching behavior in order to cache
exceptions. This cannot be done in a bridge, as the bridge might be
broken, thus preventing caching from working.

This also (and more importantly) separates concerns. The bridge should
not even care if caching is involved or not. Its purpose is to collect
and provide data.

Response times should be faster, as more complex bridge functions like
'setDatas' (evaluates all input parameters to predict the current
context) and 'collectData' (collects data from sites) can be skipped
entirely.

Notice: In its current form, index.php takes care of caching. This
could, however, be moved into a separate class (i.e. CacheAbstract)
in order to make implementation details cache specific.

* [index] Add '_error_time' parameter to $item['uri']

This ensures that error messages are recognized by feed readers as
new errors after 24 hours. During that time the same item is returned
no matter how often the cache is cleared.

References https://github.com/RSS-Bridge/rss-bridge/issues/814#issuecomment-420876162

* [index] Include '_error_time' in the title for errors

This prevents feed readers from "updating" feeds based on the title

* [index] Handle "HTTP_IF_MODIFIED_SINCE" client requests

Implementation is based on `BridgeAbstract::dieIfNotModified()`,
introduced in 422c125d8e and
simplified based on https://stackoverflow.com/a/10847262

Basically, before returning cached data we check if the client send
the "HTTP_IF_MODIFIED_SINCE" header. If the modification time is
more recent or equal to the cache time, we reply with "HTTP/1.1 304
 Not Modified" (same as before). Otherwise send the cached data.

* [index] Don't encode exception message with `htmlspecialchars`
* [Exceptions] Include error message in exception
* [index] Show different error message for error code 0
2018-10-15 17:21:43 +02:00
logmanoriginal
996295e82f Add 'dev.' to the release version in master
This helps (roughly) identifying versions when opening issues on
GitHub, using the latest ZIP file for master.

References #773
2018-09-26 20:04:27 +02:00
logmanoriginal
13bd7fe21b [contents] Return error if the server responded with any code other than 200 2018-09-26 19:16:02 +02:00
logmanoriginal
fcc9f9fd61 [FacebookBridge] Use alternative URI to load more posts
The URI "https://facebook.com/username?_fb_noscript=1" returns two
posts per user. Some profiles, however, are very active, causing the
bridge to miss items if more than two posts are send within the cache
duration (5 minutes).

The alternative suggested in #669 is to use a different URI:
"https://facebook.com/pg/username/posts?_fb_noscript=1"

While the contents of this URI essentially look the same when viewed
in a browser, it actually returns more than 10 posts depending on the
profile.

References #669
2018-09-26 18:24:46 +02:00
logmanoriginal
e1c4914b1c [FacebookBridge] Optimize for readability 2018-09-25 18:56:33 +02:00
logmanoriginal
93e7ea9fea [HtmlFormat] Make feeds available via syndication links 2018-09-22 19:51:18 +02:00
logmanoriginal
2d1b446bd1 [DevToBridge] Add new bridge
Returns feeds for tags from https://dev.to

References #840
2018-09-22 18:57:07 +02:00
logmanoriginal
1d451610d6 [ParameterValidator] Move 'getQueriedContext' from BridgeAbstract 2018-09-22 17:04:55 +02:00
logmanoriginal
f853ffc07c [ParameterValidator] Refactor 'validation' into 'ParameterValidator'
Adds a new class 'ParameterValidator' to replace the functions from
'validator.php', separating private functions from 'validateData' to
class private functions in the process.

Instead of echoing error messages, adds messages to a private variable,
accessible via 'getInvalidParameters'.

BridgeAbstract now adds invalid parameter names to the error message.
2018-09-22 16:42:04 +02:00
logmanoriginal
e3a5a6a170 [index] Update and improve parameter handling for bridge and cache
- Use 'array_diff_key' instead of 'unset'
- Remove parameters for caches

By removing certain parameters for caches, the loading times can be
improved considerably:

* action: It doesn't matter which action the user took to generate
feed items.

* format: This has the biggest impact on performance, because cached
items are now shared between different formats (i.e. try switching
between Atom, Html and Mrss and compare previous vs. now). If a
server handles lots of requests, this may even reduce bandwidth if
the same contents are requested for different formats.

* _noproxy: The proxy behavior has no impact on the produced items,
so it can be ignored.

* _cache_timeout: This is another option which might impact performance
for some servers, especially if 'custom_timeout' has been enabled in
the configuration. Requests with different cache timeouts no longer
result in separate cache files.
2018-09-22 15:44:03 +02:00
logmanoriginal
243e324efc [NineGagBridge] Fix missing sections breaking feeds
Posts may supply a list of 'sections' or a single 'postSection'

References #844
2018-09-22 15:19:14 +02:00
logmanoriginal
ae58b1566e [NineGagBridge] Remove type hinting
Type hinting for strings doesn't work prior to PHP 7, see
http://php.net/manual/en/functions.arguments.php#functions.arguments.type-declaration

References #837
2018-09-22 15:19:14 +02:00
sysadminstory
c044694b21 [ZoneTelechargementBridge] Sort episodes from newest to oldest (#835)
References #834
2018-09-21 20:22:49 +02:00
triatic
db24f55c86 [FB2Bridge] Do not strip <h3> and <h4> (#836)
Do not strip <h3> and <h4>. Output looks better when they are retained. See attached.
2018-09-21 20:19:22 +02:00
logmanoriginal
eb30038d6b [README] Update and reorganize 2018-09-16 18:20:35 +02:00
logmanoriginal
712a581ed6 [README] Add badge for Guix release
Unfortunately there is no way to query the current package version,
so this is only a placeholder
2018-09-16 16:01:51 +02:00
logmanoriginal
d3df4b51b8 [README] Add badge for current debian release 2018-09-16 15:13:30 +02:00
logmanoriginal
e6476a600d [KununuBridge] Fix broken bridge and simplify implementation 2018-09-16 09:55:35 +02:00
Grégory T
811e8d8c88 [ETTVBridge] Improvements and bug fixes (#682)
* Fix typo with status field
* Comply with other bridges

Change the uri element of an item to point, not on the magnet link, but on the page, as similar bridges do.

* Improved to return name & uri matching with query

This change makes it possible for the feed reader to discover a title and url consistent with the user's search.
2018-09-15 17:11:36 +02:00
logmanoriginal
adc6f72e97 [style] Fix first letter of labels not capitalized
This error is caused by setting label::before { content: " "; },
which makes the first letter a whitespace on all labels, neccessary
 for browsers that doesn't support the grid layout.

This commit clears the content if the browser supports the grid layout,
properly capitalizing labels again. If a browser doesn't support grid
layout, labels stay as they are provided by the bridge.
2018-09-15 17:04:20 +02:00
logmanoriginal
182153485c [Arte7Bridge] Move parameter examples into tool tip for readability 2018-09-15 16:50:10 +02:00
LogMANOriginal
bf9946d1fc CSS adjustments to improve readability for bridge parameters (#763)
* Group common selectors
* Fix indentation using tabs
* Use same styles for number and text inputs
* Use grid layout for parameters

Introduces the grid layout for bridge parameters. All parameters are
arranged in a grid to improve readability. Read more on grid layouts
at

- https://www.w3schools.com/css/css_grid.asp
- https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Grid_Layout

Notice:

Grid layouts are not supported in very old browser versions:
https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Grid_Layout/CSS_Grid_and_Progressive_Enhancement

This is why @supports checks for browser support (not supported in IE)
https://developer.mozilla.org/en-US/docs/Web/CSS/@supports#Browser_compatibility

In case grid layout is not supported, the displayed form is usable
but not very pretty due to <br> being removed by this commit for
cosmetic reasons (breaks grid layout).

Unfortunately it doesn't seem possible to insert line breaks manually
via '::after { content: '\A' }' in cases where grid layout isn't
supported.

* Add padding to card parameters

Adds padding to parameters to improve readability. For bridges without
parameters (count($parameters) === 0), the parameter 'div' is no longer
created.

* Add colon ':' after label via CSS
* Capitalize first letter of label for readability
* Fix checkbox isn't aligned left

Sets the size of the checkbox to 20x20 px for good measure.

* Harmonize formatting
* Add new style to number and select boxes

References #797

* Add fallback solution for browsers without grid support
2018-09-15 16:39:50 +02:00
triatic
ec60752650 [FB2Bridge] Prevent Facebook link href's ending in two quotes (#831)
Additionally prevent Facebook links having two forward slashes after the hostname.
2018-09-15 15:16:15 +02:00
sysadminstory
6688cf0c3b [AutoJMBridge] Fix concatenation bug (#833) 2018-09-15 15:12:34 +02:00
ORelio
ae45a8cfee [contents] Fix open_basedir warning (#832)
References #818
2018-09-15 14:46:11 +02:00
Matthew Seal
e34ef6cb4f [MrssFormat] Escape double quotes in XML attributes (#813)
XML attributes need to have certain characters escaped to be valid. The title attribute can have double quotes in it which need to be properly encoded for attributes.
2018-09-15 14:13:05 +02:00
sysadminstory
5c92a736fa [ZoneTelechargementBridge] Added Bridge for ww2.zone-telechargement1.org (#829)
* [ZoneTelechargementBridge] Added Bridge for ww2.zone-telechargement1.org

Goal for this bridge is to follow the episode publication of a TV show
season while it's broadcasted on the TV.
2018-09-13 19:36:48 +01:00
Eugene Molotov
911bcfb246 [PikabuBridge] Implemented bridge (#830)
* [PikabuBridge] Implemented bridge
2018-09-13 12:52:26 +01:00
ZeNairolf
efa550ef61 Add 9gag.com bridge (#801)
* Add 9gag.com bridge
2018-09-13 10:11:42 +01:00
sysadminstory
d5d7683ed3 [AutoJMBridge] New Bridge (#827)
* [AutoJMBridge] New Bridge

This bridge will show all the car offers AutoJM has for the model you
choosed and using your filter. Very useful to wait for a cheap price for
a new car !
2018-09-13 10:05:07 +01:00
triatic
fe94914eb5 [AtomFormat.php] Eliminate PHP Notice when running in CLI mode (#824) 2018-09-12 14:37:27 +01:00
Quentin Delmas
622802e5d4 Fix multiple warnings.
Fix JSON request string in case of empty location
2018-09-12 13:31:11 +01:00
sysadminstory
6da8daf1a3 [DealabsBridge] Fix for #782 and all categories are now available (#821)
This commit fixes #782 by updating the parameter value of 'Maison &
Jardin', but this means the user has to update his RSS Feed URL (.because
of the bridge structure, it would be a nightmare to fix it in another
way)

This commits add all the categories available on Dealabs Website.
2018-09-11 22:11:00 +01:00
la Bécasse
654e502e84 Arte7 collection support (#819)
* Arte7 collection support
2018-09-11 22:09:47 +01:00
sysadminstory
c8ace9e3bd [ExtremeDownloadBridge] Added Bridge for ww1.extreme-d0wn.com (#820)
* [ExtremeDownloadBridge] Added Bridge for ww1.extreme-d0wn.com

Goal for this bridge is to follow the episode publication of a TV show season
while it's broadcasted on the TV.
2018-09-11 20:10:46 +01:00
Monsieur Poutounours
5722a6c139 Adding a bridge for theyetee.com (#809)
* Adding a bridge for theyetee.com

The bridge fetches daily shirts at theyetee.com.
The Yetee offers two new shirts each day, but you can buy them only for a few hours !
Unfortunately, the site don't provide RSS feed, so the only way to keep up to date on new shirt is their daily mailing ... until now !
2018-09-10 20:56:55 +01:00
Quentin Delmas
458b826871 Remove declaration of extractFromDelimiters, it is now a reusable function. Fixes #815 2018-09-10 09:29:19 +01:00
Quentin Delmas
b397a42876 version: Bump to 2018-09-09 2018-09-09 21:00:10 +01:00
Corentin Garcia
111c45d010 [GithubSearchBridge] Fix content parsing, add tags if present (#803)
* [GithubSearchBridge] Fix content parsing, add tags if present

* [GithubSearchBridge] Add categories (from tags)
2018-09-09 20:30:29 +01:00
Corentin Garcia
55b36b0455 [DauphineLibereBridge] Use https, fix content parsing (fix issue #780) (#811) 2018-09-09 20:23:59 +01:00
ORelio
de8cee6a1c Catching up | [Main] Debug mode, parse utils, MIME | [Bridges] Add/Improve 20 bridges (#802)
* Debug mode improvements

 - Improve debug warning message
 - Restore error reporting in debug mode
 - Fix 'notice' messages for unset fields

* Add parsing utility functions

html.php
 - extractFromDelimiters
 - stripWithDelimiters
 - stripRecursiveHTMLSection
 - markdownToHtml (partial)

bridges
 - remove now-duplicate functions
 - call functions from html.php instead

* [Anidex] New bridge

Anime torrent tracker

* [Anime-Ultime] Restore thumbnail

* [CNET] Recreate bridge

Full rewrite as the previous one was broken

* [Dilbert] Minor URI fix

Use new self::URI property

* [EstCeQuonMetEnProd] Fix content extraction

Bridge was broken

* [Facebook] Fix "SpSonsSoriSsés" label

... which was taking space in item title

* [Futura-Sciences] Use HTTPS, More cleanup

Use HTTPS as FS now offer HTTPS
Clean additional useless HTML elements

* [GBATemp] Multiple fixes

- Fix categories: missing "break" statements
- Restore thumbnail as enclosure
- Fix date extraction
- Fix user blog post extraction
- Use getSimpleHTMLDOMCached

* [JapanExpo] Fix bridge, HTTPS, thumbnails

- Fix getSimpleHTMLDOMCached call
- Upgrade to HTTPS as JE now offers HTTPS
- Restore thumbnails as enclosures

* [LeMondeInformatique] Fix bridge, HTTPS

- Upgrade to HTTPS as LMI now offers HTTPS
- Restore thumbnails using small images
- Fix content extraction
- Fix text encoding issue

* [Nextgov] Fix content extraction

- Restore thumbnail and use small image
- Field extraction fixes

* [NextInpact] Add categories and filtering by type

- Offer all RSS feeds
- Allow filtering by article type
- Implement extraction for brief articles
- Remove article limit, many brief articles are publied all at once

* [NyaaTorrents] New bridge

Anime torrent tracker

* [Releases3DS] Cache content, restore thumbnail

- Use getSimpleHTMLDOMCached
- Restore thumbnail as enclosure

* [TheHackerNews] Fix bridge

 - Fix content extraction including article body
 - Restore thumbnail as enclosure

* [WeLiveSecurity] HTTPS, Fix content extraction

- Upgrade to HTTPS as WLS now offers HTTPS
- Fix content extraction including article body

* [WordPress] Reduce timeout, more content selectors

- Reduce timeout to use default one (1h)
- Add new content selector (articleBody)
- Find thumbnail and set as enclosure
- Fix <script> cleanup

* [YGGTorrent] Increase limit, use cache

- Increase item limit as uploads are very frequent
- Use getSimpleHTMLDOMCached

* [ZDNet] Rewrite with FeedExpander

- Upgrade to HTTPS as ZD now offers HTTPS
- Use FeedExpander for secondary fields
- Fix content extraction for article body

* [Main] Handle MIME type for enclosures

Many feed readers will ignore enclosures (e.g. thumbnails) with no MIME type. This commit adds automatic MIME type detection based on file extension (which may be inaccurate but is the only way without fetching the content).

One can force enclosure type using #.ext anchor (hacky, needs improving)

* [FeedExpander] Improve field extraction

- Add support for passing enclosures
- Improve author and uri extraction
- Fix 'notice' PHP error messages

* [Pull] Coding style fixes for #802

* [Pull] Implementing changes for #802

 - Fix coding style issues with str append
 - Remove useless CACHE_TIMEOUT
 - Use count() instead of $limit
 - Use defaultLinkTo() + handle strings
 - Use http_build_query()
 - Fix missing </em>
 - Remove error_reporting(0)
 - warning CSS (@LogMANOriginal)
 - Fix typo in FeedExpander comment

* [Main] More documentation for markdownToHtml

See #802 for more details
2018-09-09 20:20:13 +01:00
Quentin Delmas
123fce4394 [ForGifsBridge] Fix permissions of ForGifsBridge 2018-09-09 17:34:36 +01:00
Quentin Delmas
a3f99c9c3f [GOGBridge] Added bridge for GOG.com 2018-09-09 17:32:36 +01:00
Eugene Molotov
bf30ad127c [FacebookBridge] Removes query string from post links
* [FacebookBridge] Removes query string from post links
2018-09-09 16:31:15 +01:00
logmanoriginal
37f84196b7 [GooglePlusPostBridge] Fix title is empty if content is too short
The bridge would generate empty titles if the content is longer than
50 characters, but doesn't have further spaces in it. With this commit
the title is correctly generated based on the contents, taking missing
spaces into account.

References #786
2018-09-08 17:07:57 +02:00
Corentin Garcia
44764f7182 [GrandComicsDatabaseBridge] Fix links in content (#804) 2018-09-08 11:12:27 +01:00
Antoine Cadoret
19f294d71d Add fields to leboncoin bridge (#783)
* [LeBonCoinBridge] Add fields to LeBonCoinBridge
2018-08-31 14:34:41 +01:00
Teromene
b0e33e4e01 Update LeBonCoinBridge to use the site's API (#795)
* Update LeBonCoinBridge to use the site's API
2018-08-28 14:20:02 +01:00
Eugene Molotov
558fa50a2a [core] Enabled debug mode before including core files (#790) 2018-08-25 20:02:47 +01:00
Eugene Molotov
ffb8b82c73 [FileCache] reseting cached file stat result to have correct getTime() result (#792)
* [FileCache] reseting cached file stat result to have correct getTime() result
2018-08-25 20:00:51 +01:00
Eugene Molotov
422c125d8e [core] Returning 304 http code when returning cached data (#793) 2018-08-25 20:00:38 +01:00
Quentin Delmas
059656c370 Fix phpcs. 2018-08-22 16:25:08 +01:00
Quentin Delmas
9fc1e97efe Avoid bot exclusion. 2018-08-22 16:21:39 +01:00
Quentin Delmas
be3620acb7 Add extension check for the "json" extension. 2018-08-22 16:21:20 +01:00
LogMANOriginal
16c0a61232 [README] Add a "Deploy to Cloud" button for Docker
Adds a button to deploy RSS-Bridge to the Docker Cloud as described here: https://docs.docker.com/docker-cloud/apps/deploy-to-cloud-btn/
2018-08-21 18:40:39 +02:00
Walter Barrett
704a87ad97 Icons: Allow Bridge-specified icons (#788) 2018-08-21 17:46:47 +02:00
sysadminstory
c4cccfe0f3 [LesJoiesDuCode] Switch to HTTPS and remove author (#787)
Website offers now HTTPS, therefore the bridge was switched to it.
The post author is not displayed anymore on the homepage, so it has been
removed.
2018-08-21 17:41:56 +02:00
Marcin C
d07deb0930 css: Modern look for RSS-Bridge (#781) 2018-08-21 17:22:46 +02:00
Piranhaplant
e7dab5d351 Fixed timestamp on Pixiv bridge (#785) 2018-08-18 16:54:24 -03:00
logmanoriginal
ad82d50bbd [CNETBridge] Remove bridge
CNET now provides public feeds at https://www.cnet.com/rss/

References #775
2018-08-12 11:02:44 +02:00
logmanoriginal
c305c1ded7 [BlaguesDeMerdeBridge] Adjust to layout changes
References #767
2018-08-10 21:08:47 +02:00
logmanoriginal
f14a5bd771 [CADBridge] Remove bridge
https://cad-comic.com/ now provides feeds at

- https://cad-comic.com/feed (rss)
- https://cad-comic.com/feed/atom (atom)

Thus multiple alternatives are available to choose from, making this
bridge obsolete:

- FilterBridge (using one of the feeds above)
- WordPressBridge (on the main site)
- One of the two available feeds

References #752
2018-08-10 19:53:32 +02:00
logmanoriginal
a20d5f9af0 tests: reuse RssBridge.php instead of implementing a custom solution 2018-08-10 15:33:32 +02:00
logmanoriginal
ee28b124e0 [DanbooruBridge] Fix bridge
This commit fixes an issue caused by self closing tags not supported
by simplehtmldom (<source>).

Adds a monkey patch to extend simplehtmldom with the ability to detect
that particular tag. Most of the code added is copied directly from
simplehtmldom (see vendor/simplehtmldom) with adjustments to account
for RSS-Bridge formatting.

Related to: https://sourceforge.net/p/simplehtmldom/bugs/83/

Notice: The tag itself is valid according to Mozilla:

The HTML <picture> element serves as a container for zero or more
<source> elements and one <img> element to provide versions of an
image for different display device scenarios. The browser will
consider each of the child <source> elements and select one
corresponding to the best match found; if no matches are found
among the <source> elements, the file specified by the <img>
element's src attribute is selected. The selected image is then
presented in the space occupied by the <img> element.

-- https://developer.mozilla.org/en-US/docs/Web/HTML/Element/picture

References #753
2018-08-09 21:55:43 +02:00
LogMANOriginal
7dee3a175a [index] Add '?action=list' to list bridges (#493)
Adds a new action '?action=list' to return a list of bridges as JSON formatted text. Each bridge brings following information:

- status (active/inactive)
- uri
- name
- parameters
- maintainer
- description

For inactive bridges only the status is returned.
Bridges that cannot be instantiated are considered inactive.
2018-08-09 19:14:10 +02:00
logmanoriginal
5fea9fc1f5 bridges: Fix bridges failing unit test 2018-08-09 17:04:16 +02:00
logmanoriginal
6bceb2b2db [tests] Add unit test for bridge implementation
Adds unit test for bridge implementations:

- Custom functions must be in protected or private scope
- getName() must return a valid string (non-empty)
- getURI() must return a valid URI
- Each bridge must define constants for NAME, URI, DESCRIPTION and
  MAINTAINER. CACHE_TIMEOUT and PARAMETERS are optional.

The unit test is written for PHPUnit 6.x and will automatically be
tested by Travis-CI for PHP 7.0 (see .travis.yml).

Remarks:

Unit tests for bridge data were scrapped in #378 for complexity
reasons (tests would have to be maintained for each bridge). This
unit test, however, is written for testing all bridges without
taking specific implementation details into account.
2018-08-09 17:04:16 +02:00
Eugene Molotov
df81fa62d1 [VkBridge] Video attachment fixes (#766)
* use defaultLinkTo
* remove duplicate video links
* remove line ending before "Reposted" label
* return newline before reposted string
* remove comments
* use video links that won't require login
* set title if video has no title
2018-08-09 17:02:36 +02:00
Eugene Molotov
f8c6400373 [HtmlFormat] Hide "Categories" label, if array of categories is empty (#765) 2018-08-09 16:46:53 +02:00
logmanoriginal
de7622ebbf version: Bump to 2018-08-07 2018-08-07 18:37:38 +02:00
logmanoriginal
09c9d015b4 [ForGifsBridge] Add new bridge 2018-08-04 23:42:58 +02:00
logmanoriginal
3a496e3b18 [FilterBridge] Add option to build title from content
Adds a new option '&title_from_content=on' to build the title for feed
items from the feeds content. The title is generated from the first
whitespace after 50 characters of the content or the entire content if
the total size is lower than 50 characters.

References #587
2018-08-04 20:46:59 +02:00
Eugene Molotov
df58f5bbdb [core] Add urljoin (#756)
Adds php-urljoin from https://github.com/fluffy-critter/php-urljoin to replace the custom implementation of 'defaultLinkTo'
2018-08-02 06:31:56 +02:00
logmanoriginal
9d0452d11b [.travis] Use composer for HHVM
This fixes the HHVM build failing because pear doesn't exist in HHVM.
2018-08-01 19:37:10 +02:00
sublimz
f92ac49947 [LeBonCoinBridge] Add cities support (#751) 2018-08-01 17:25:18 +02:00
Benasse
a574fa15ac [YGGTorrentBridge] Order search result by publish date (#762) 2018-07-31 21:46:10 +02:00
Nemo
8f9a385b4d [AmazonPriceTrackerBridge] Improve Amazon scraper logic (#761)
- Now works on all websites, and even with products
  with multiple prices
- Closes #750
2018-07-31 21:44:37 +02:00
logmanoriginal
53bdfa3bf0 [GooglePlusPostBridge] Skip posts without message 2018-07-31 19:15:09 +02:00
logmanoriginal
53278b2eed [GooglePlusPostBridge] Add option to include image in content
References #600
2018-07-31 19:09:12 +02:00
logmanoriginal
5f3c55b808 [GooglePlusPostBridge] General cleanup 2018-07-31 18:55:35 +02:00
logmanoriginal
fb79a67370 [GooglePlusPostBridge] Normalize static::URI usage
This commit fixes a few things related to static::URI

1) Remove trailing slash from the URI to simplify using 'defaultLinkTo'
2) Use static::URI instead of self::URI for consistency
3) Remove custom implementation of 'defaultLinkTo'
2018-07-31 18:29:14 +02:00
logmanoriginal
3c4e12ceba [GooglePlusPostBridge] Add images to enclosures
Images are collected for each post and added to enclosures. Images or
animtions from lh3.googleusercontent.com are specifically handled in
order to return the animated version of the gif and the original sized
image (this is normally taken care of by JS in the browser).
2018-07-31 18:18:22 +02:00
logmanoriginal
0d1923c52f [GitHubGistBridge] Add new bridge
Adds a new bridge for https://gist.github.com

The bridge generates feeds for comments on a particular gist based on
the gist ID or full URI. For better readability the general behavior
of code sections is manually restored with the original CSS styles
from GitHub.
2018-07-29 16:31:47 +02:00
logmanoriginal
ce896b4247 [SkimfeedBridge] Add new bridge
New bridge for Skimfeed: https://skimfeed.com

Generates feeds for all features of Skimfeed:

- News (the ones displayed on the front page)
- Hot topics ("What's Hot" section on the front page)
- Tech news (preconfigured feeds in the menu bar)
- Custom feeds (using the configuration system of Skimfeed), see
https://skimfeed.com/custom.php

The number of items returned by the bridge can be limited for all
categories ('&limit=...'). This parameter is optional, all categories
are unlimited by default!

Authors are added with HTML anchors in order to allow quick navigation
to source channels.

The bridge ships with developer tools to auto-generate lists in the
future (especially useful for 'Tech news'!)

References #748
2018-07-27 23:18:32 +02:00
sysadminstory
a4b2d88dbe [DealabsBridge] Follow website change (#758) 2018-07-25 20:02:31 +02:00
logmanoriginal
65ec04ea98 [contents] Remove superfluous debug log from getContents
References #757
2018-07-25 19:56:46 +02:00
logmanoriginal
afb4de318b [FlickrBridge] Fix missing scheme for image URLs
References #754
2018-07-23 20:14:46 +02:00
Eugene Molotov
43bb17f995 [VkBridge] Converting hashtags to categories (#755)
* [VkBridge] Converting hashtags to categories
2018-07-22 16:43:00 +02:00
logmanoriginal
bae7a5879f [FlickrBridge] Fixed broken bridge
Following changes in the JSON data and selecting images for the
content (320x240 or bigger) and enclosure (largest version). All of
the data is now extracted from the JSON data instead of parsing the
DOM.

References #754
2018-07-22 14:06:04 +02:00
LogMANOriginal
bd760cbcee [README] Add docker build status 2018-07-21 21:59:48 +02:00
LogMANOriginal
cd20b4476f [README] Add label for latest release 2018-07-21 21:54:46 +02:00
LogMANOriginal
d83f2f285b Separate index and bridge card generating code into a separate classes (#734)
[html] Generate index and bridge cards using separate clases

Move HTML generating code from 'index.php' to 'Index.php', separating components into static functions.

Move HTML generation code for bridge cards from 'html.php' to 'BridgeCard.php', separating components into static functions.
2018-07-21 18:15:07 +02:00
logmanoriginal
15e6d77569 [FierPandaBridge] Fix bridge
This bridge now returns all articles from the front page, following
layout changes in the past.

References #679
2018-07-21 18:07:03 +02:00
logmanoriginal
f97d2ef254 [Torrent9Bridge] Remove bridge
The site moved from www.torrent9.pe to www.t9.pe and is now protected
by Cloudflare challenges, making it inaccessible to RSS-Bridge.
2018-07-21 17:45:22 +02:00
logmanoriginal
91ae2a23d7 [CpasbienBridge] Remove bridge
Removing this bridge for two reasons:

1) The service moved from www.cpasbien.cm to www.torrents9.blue,
changing the layout in the process (incompatible).

2) The new site is permanently protected by Cloudflare IUAM, making
it inaccessible by RSS-Bridge.

While it would certainly be possible to rewrite the bridge to work
with the new layout, the site is still inaccessible.

References #605
2018-07-21 17:43:29 +02:00
logmanoriginal
066ef1d7db [contents] Add Cloudflare challenge detection
Adds detection for servers responding with Cloudflare challenges,
throwing a server error if detected:

"The server responded with a Cloudflare challenge, which is not
supported by RSS-Bridge! If this error persists longer than a week,
please consider opening an issue on GitHub!"

This is supposed to support maintainers to identify broken bridges
for sites with Cloudflare enabled permanently. It doesn't circumvent
the protection in any form or shape!

The Cloudflare challenge is detected by analyzing the last response
header received from the server. If the HTTP Code is not 200 (OK)
and the server name contains 'cloudflare' ('Server: cloudflare'),
RSS-Bridge assumes the server responded with a challenge.

The header parsing is based on https://stackoverflow.com/a/18682872
2018-07-21 17:43:29 +02:00
LogMANOriginal
4facbf32e3 [InstructableBridge] Add new bridge (#724)
This commit adds a new bridge for http://www.instructables.com. This bridge
currently supports fetching content by category (all categories available 200+),
using available filters (featured, recent, popular, views, contest winners).
2018-07-21 15:25:13 +02:00
logmanoriginal
6bd76af326 [YoutubeBridge] Add duration limits for all modes
Adds duration limits (minimum duration, maximum duration) for all
modes (user/id/playlist/search). Duration limits are optional, so
existing subscriptions don't break.

The limits are specified by two separate parameters, each of which
is optional:

- `&duration_min=` (minimum duration in minutes, default: -1)
- `&duration_max=` (maximum duration in minutes, default: INF)

If duration limits are specified in either user, id or playlist mode,
the bridge defaults to fetching data from HTML intead of XML feeds,
which requires more bandwidth and takes longer, because each video is
loaded individually!

References #670
2018-07-21 14:33:07 +02:00
logmanoriginal
caa622ffec [search] Support searching by URI
Adds matching for URIs to the search bar, using the format
<scheme>://<host>/<path>

Searching by URI scheme is also supported:

"http://"  (returns all bridges with 'http'  scheme)
"https://" (returns all bridges with 'https' scheme)

The following examples are equivalent and will return both of the
Facebook bridges (FacebookBridge and FB2Bridge):

"https://www.facebook.com/facebook"
"https://www.facebook.com/facebook?..."
"https://www.facebook.com"
"http://www.facebook.com"
"https://facebook.com"
"http://facebook.com"
"facebook.com"
"facebook"

Notice: When the URI scheme is omitted, the search algorithm falls back
to regex matching. Searching for "www.facebook.com" doesn't work, as it
is missing the schema and doesn't match via regex!

Omitting the 'www.', however, does work. This was a design decision for
some bridges specify their URI with and others without 'www.'

A search term can still be specified in the browser URL using parameter
'q' => '?q=searchterm'.

References #743
2018-07-20 22:44:13 +02:00
teromene
c4d489f018 Add URI to ElloBridge elements. 2018-07-19 17:07:54 +02:00
logmanoriginal
6a98293fb3 [Configuration] Bump version to 2018-07-17 2018-07-17 20:44:01 +02:00
logmanoriginal
d79630e3b8 [Configuration] Remove check for allow_url_fopen
This commit follows the changes done in commits

fbf874cb29
ead7b2e8de
2018-07-17 20:39:14 +02:00
teromene
1f2fe25471 Fix LeBonCoinBridge, now uses getContents correctly, 2018-07-17 10:50:30 +02:00
Antoine Cadoret
87fc9e9156 fix LeBonCoin bridge (#747) 2018-07-16 20:13:08 +02:00
Nemo
c7b0c9fd31 Amazon Price Tracker Bridge (#741)
* [amazonprice] Adds AmazonPriceTracker bridge
2018-07-16 14:54:52 +02:00
Teromene
fbf874cb29 Update README.md
Remove allow_url_fopen requirement. This should no longer be necessary. Added requirement for curl.
2018-07-16 12:37:09 +02:00
Eugene Molotov
049ee52fb5 Implemented feed item categories (#746) 2018-07-16 12:32:24 +02:00
TheRadialActive
3f41d0593a Added RSS bridge for zenodo.org (#749)
* added RSS bridge for zenodo.org
2018-07-16 12:02:41 +02:00
sysadminstory
7126f5e838 [DealabsBridge] First version of the generic "Pepper" Bridge (#726)
* [DealabsBridge] First version of the generic "Pepper" Bridge
2018-07-13 00:35:13 +01:00
Nemo
ead7b2e8de [fb2] Switches to getContents (#742) 2018-07-10 02:29:47 +01:00
LogMANOriginal
0d80a19e84 [FacebookBridge] Add context for public Facebook groups (#739)
The previous context is now labeled 'User', while the new context is
labeled 'Group'. The existing code was not changed, instead new group*
functions were implemented to handle groups.

The general principle of capturing groups is the same as done for users
with adjustments to account for different HTML structures.

Captcha responses are currently not supported for groups! There doesn't
seem to be a way to trigger them consistently, which makes it hard to
handle them properly.

Features of the group context:

- The feed title is based on the group name
- The group URI used for capturing is returned for the feed URI
- Author names and timestamps are reproduced from the source
- Post titles are reproduced from the source if they exist, otherwise
the title is build manually from the author name and the content
- Original contents are included with the feed
- All images are attached as enclosures as well

Closes #
2018-07-08 17:16:00 +02:00
logmanoriginal
42c699f474 formats: Fix favicon not found if url contains path 2018-06-30 10:27:05 +02:00
logmanoriginal
2bc8daa101 [JustETFBridge] Add new bridge
Supports latest news and profiling a given ETF in Englisch, German
or Italian language. Cover images are attached as enclosures and not
as part of the content.

News:

Optionally loads the full article for each news item. Some articles
may include scripts to provide interactive graphs. These scripts are
removed as they would be rendered as pure text and a message is shown
instead: "[Content removed! Visit site to see full contents!]"

Profile:

Optionally includes the ETF strategy and description.
2018-06-30 10:27:05 +02:00
logmanoriginal
bca79d3f88 [KununuBridge] Fix broken page layout and sort reviews 2018-06-30 10:27:05 +02:00
logmanoriginal
90dc968fd1 Fix PHPCS error 2018-06-30 10:26:48 +02:00
Teromene
da6b98851c Add recuperation of the current version from git if available (#731)
* Add recuperation of the current version from git if available
* Include version when auto-reporting an error
2018-06-30 10:24:22 +02:00
teromene
71c29d4192 Fix phpcs for master. 2018-06-29 23:15:22 +01:00
LogMANOriginal
193ca87afa [phpcs] enforce single quotes (#732)
* [phpcs] Add rule to enforce single quoted strings
2018-06-29 22:55:33 +01:00
Nemo
5ea79ac1fc Add markdown support to Container Linux Feed (#730) 2018-06-28 20:54:42 +02:00
Teromene
937ea49271 Add basic authentication support (#728)
* Move configuration in its own class in order to reduce the verbosity of index.php
* Add authentication mechanism using HTTP auth
* Add a method to get the config parameters
* Remove the installation checks from the index page
* Log all failed authentication attempts
2018-06-27 19:09:41 +02:00
logmanoriginal
95686b803c [IsoHuntBridge] Remove bridge
isoHunt has discontinued services due to legal reasons and is now
accessible via https://isohunts.to

While it is certainly possible to rewrite the bridge to fetch some
information from the new site, it wouldn't be able to provide as
much functionality as before. This is due to isoHunt having removed
all searching and filtering options, only providing static HTML pages
for general categories (anime, movies, etc...). Those pages, however,
are heavily broken.

Unless someone is interested in monitoring the general categories
the effort of upgrading the bridge to the new site is not worth taking
time for.

Users of isoHunt are asked to make use of their client application,
as they don't provide online services anymore (it's now in the darknet)

Here is the statement from isoHunt:

"Due to hard regulations and security issues for bittorrent users, we
have moved into a more secure and even faster district of the internet!

[...]

Torrent Downloads have a high risk of getting legal problems. That is
why we do not offer torrentfiles any more. [...]"

-- source: https://isohunts.to
2018-06-24 18:33:50 +02:00
logmanoriginal
5087f5f79e [FacebookBridge] Support facebook links as user name
Allows users to paste facebook links as user name. The link must contain
the correct host (www.facebook.com) and a valid path (/user-name/...).
The first part of the path is used for the user name. Errors are returned
in case something went wrong.

References #706
2018-06-24 11:14:08 +02:00
logmanoriginal
4a5f190e0e [FacebookBridge] Add option to skip reviews
Reviews are provided the same way as summary posts and therefore returned
as separate feed item for each review. This commit adds a new option
'&skip_reviews=on' to skip reviews entirely.

References #706
2018-06-24 10:52:22 +02:00
logmanoriginal
01a2746715 [YoutubeBridge] Fix sniff violation
This is a fix for a sniff violation not detected by newer versions
of phpcs (not sure why though, it's detected in version 2.7.1).
2018-06-23 21:28:30 +02:00
Nemo
f4a60c1777 Add dockerfile to create an official docker image (#720) 2018-06-23 16:51:48 +02:00
sysadminstory
1b08bce779 [DealabsBridge] Follow site changes (#721)
- Changed some CSS class to follow the website changes (again)
2018-06-21 13:14:59 +01:00
Nemo
9fa74a36c6 Adds Container Linux releases RSS Feed (#718)
* Adds Container Linux releases RSS Feed
2018-06-19 19:39:08 +01:00
Corentin Garcia
7493e2b5b8 [GrandComicsDatabaseBridge] Add bridge (#717)
closes #709
2018-06-15 21:09:09 +02:00
Corentin Garcia
8e468a9ca7 [SuperSmashBlogBridge] Added bridge (#716) 2018-06-15 21:05:31 +02:00
Joe Digilio
50924b9213 Abort on parse error of config.default.ini.php (#714)
If there is an error parsing the default config file, then abort.
2018-06-15 21:02:06 +02:00
logmanoriginal
4c5013bc82 [index] Bump release version to 2018-06-10 2018-06-10 22:14:58 +02:00
Eugene Molotov
7dc09db9ca [VkBridge] More beatifications and fixes (#712)
* Add one more selector for article_author_selector
* Extend video parsing
* Add poll parsing
2018-06-10 22:09:50 +02:00
hunhejj
d92da8f0f7 Add cUrl error message and code to the debugMessage (#711) 2018-06-10 22:08:45 +02:00
logmanoriginal
064ba456e8 [InstagramBridge] Fix broken compatibility for media_type parameter
The media_type parameter was recently replaced by media_type_u (for
user mode) and media_type_h (for hashtag mode). This was necessary
in order to add the media type 'story' only for the user mode.

"The reason for that is that RSS-Bridge supports multiple parameters
with the same name if and only if they contain the exact same value.
Here, hashtags don't have stories, so it would not be possible to
pass "story" as a parameter. This is a design mistake that I made
when I added support for hashtags."

-- 8770c87389 (r28871502)

However as pointed out this change breaks existing feeds as the
parameter name is no longer compatible to previous implementations.

This commit changes the implementation to provide the old media_type
parameter globally and check for invalid options on each request. If
a user uses the 'story' option in history mode the bridge returns a
client error.

references 8770c87
references #694
fixes #696
fixes #699
fixes #701
2018-05-29 12:52:31 +02:00
LogMANOriginal
8ac8e08abf Add user config (#653)
Uses the parse_ini_file function to load default settings from the default configuration file 'config.default.ini.php'. Optionally loads custom settings from 'config.ini.php' to replace the default
values.
2018-05-29 11:52:17 +02:00
rogerdc
c4f32c31a8 Add ChristianDailyReporterBridge (#697) 2018-05-29 11:28:22 +02:00
Eugene Molotov
4369e077c2 [VkBridge] Fixed image src link generating for photo (#700) 2018-05-29 11:01:54 +02:00
sysadminstory
1045850043 [DealabsBridge] Follow site changes, fix unhandled case (#703)
* [DealabsBridge] Follow site changes, fix unhandled case

- Fixed the case where no discount was shown
- Changed some CSS class to follow the website changes
2018-05-29 10:52:13 +02:00
teromene
2d8f4dc3c5 Fix space in URL resulting in API errors. 2018-05-05 18:10:19 +01:00
teromene
779b638fb4 Added ElloBridge. Closes #683 2018-05-05 18:06:27 +01:00
teromene
3ca59392c2 Fix for crashes when accessing FileCache in case it has been purged/not created yet. 2018-05-05 18:05:48 +01:00
teromene
9b34b68180 Do not use an external service in order to fetch the favicon. 2018-05-05 13:55:38 +01:00
teromene
79ebdc4b39 Warn the user when trying to fetch a non-public facebook page. 2018-05-05 13:49:49 +01:00
teromene
8770c87389 Added support for stories in InstagramBridge. Closes #665
Renamed parameters as stories are only available in user mode.
Use a regex instead of HTML parsing to extract the JSON, as it is way faster.
2018-05-05 13:00:59 +01:00
Eugene Molotov
c1e3352218 [VkBridge] Extended article link parsing (#685)
* [VkBridge] Extended article link parsing
2018-05-05 12:03:54 +02:00
Grégory T
00570ce1b4 [ETTVBridge] New bridge, first push (#680)
* [ETTVBridge] New bridge
2018-04-30 23:18:39 +02:00
teromene
df33dcff4e [YGGTorrentBridge] URL encode the first parts of the requests. 2018-04-26 22:57:18 +01:00
Nicolas Delsaux
e60b5ab193 Mise à jour du bridge pour WorldOfTanks (#527)
* Mise à jour de l'un de mes bridges fétiches
2018-04-22 12:58:07 +02:00
teromene
b0c7a62f74 [index] Bumped version to 2018-04-20 2018-04-20 17:15:25 +02:00
teromene
57b15a089e Added DiscogsBridge. Closes #615 2018-04-20 16:57:09 +02:00
teromene
4b7fbe4188 DansTonChatBridge: test before accessing plaintext 2018-04-19 21:00:18 +02:00
Teromene
2390fb58b3 Merge pull request #673 from GregThib/patch-1
DansTonChatBridge: Update to follow DTC website changes
2018-04-19 20:58:01 +02:00
Teromene
1e8d29f6ec Merge pull request #672 from em92/patch-3
[YoutubeBridge] Removed duration in titles on search mode
2018-04-19 20:56:34 +02:00
Eugene Molotov
644d13686c [YoutubeBridge] Removed duration in titles on search mode 2018-04-19 09:03:29 +05:00
teromene
aa0ff1c9b1 Added YGGTorrentBridge. 2018-04-18 21:57:27 +02:00
teromene
539d9f1f06 Add SupInfoBridge, fixes #668 2018-04-18 12:39:45 +02:00
teromene
5ece801ce7 Fix h* display size in HtmlFormat, and fix images being wider than the page. 2018-04-18 12:29:22 +02:00
GregThib
4dcea6d9c9 Update to follow DTC website changes
Now, entry title is optionnal and may be found in h3 HTML element.
Entry content is mandatory and may be found in div[class="item-content"] HTML element.

Moreover, the title may contain simple quotes (here, encoded) so the bridge have to decode first to apply format library function. In case we don't do that, the format function double encode the quote and something like &amp;#039; could appear.
2018-04-18 12:00:00 +02:00
teromene
d69e2521f1 Removed T411 bridge. Website was closed nearly one year ago. 2018-04-18 11:44:54 +02:00
teromene
7927d73719 Rewrote DemonoidBridge. Fixes #626. 2018-04-17 15:25:02 +02:00
teromene
0620f30ae0 Changed the API key used for SoundCloud bridge. Should fix #599 2018-04-17 14:24:00 +02:00
teromene
795494cfce Added enclosures to InstagramBridge. 2018-04-16 19:34:21 +02:00
teromene
ba8542156c Remove usage of function file_get_contents. 2018-04-16 19:27:20 +02:00
Eugene Molotov
55f112e034 [VkBridge] Rewrited bridge code (#667)
* [VkBridge] Convert special HTML entities to characters in pageName

* [VkBridge] Generate feed item title

* [VkBridge] Remove double backslashes in feed item link

* [VkBridge] Unpin post if pinned

* [VkBridge] Mark reposted messages

* [VkBridge] Correct external link parsing

* [VkBridge] Added article parsing

* [VkBridge] Added video parsing

* [VkBridge] Added photo parsing

* [VkBridge] Added album link parsing

* [VkBridge] Added one more external link selector

* [VkBridge] Using array of link selectors to remove

* [VkBridge] Added document parsing

* [VkBridge] Added sign parsing

* [VkBridge] Fixed incorrect sorting with pinned item

* [VkBridge] More methods to parse documents

* [VkBridge] Save fallback if page name element not found

* [VkBridge] Using post signed as feed item author

* [VkBridge] Fixed document link

* [VkBridge] Coding policy fixes
2018-04-16 10:55:31 +01:00
Mitsukarenai
208fff801d [FDroid] minor fixes for Travis CI 2018-04-15 13:21:48 +02:00
Mitsukarenai
3c9860de43 [FDroid] new bridge 2018-04-15 13:13:10 +02:00
Adam Tygart
a16ec196c5 [NotAlways] Add a bridge for the NotAlways family of sites (#537)
NotAlways right found it necessary to remove their RSS feeds recently. This is a *simple* bridge to grab the ones on the front page. It allows you to filter the articles based on their classification (right, working, romantic, related, learning, friendly, hopeless, unfiltered, or all).
2018-04-15 12:02:37 +01:00
teromene
887fc7b037 Fix GoComics, website completely changed. Fixes #663 2018-04-14 18:15:44 +01:00
teromene
1bd4a40f71 Added GNOME Builder configuration to gitignore. 2018-04-14 18:15:13 +01:00
teromene
494169f959 Added bridge for Pixiv.
This bridge is slow, as caching of images is required (REFERER header required to access the full size images)
2018-04-14 16:19:35 +01:00
logmanoriginal
178177e787 [index] Push version to 2018-04-06 2018-04-06 22:45:33 +02:00
logmanoriginal
1cb83ccea3 [IPBBridge] Use limit for the number of items
The limit was used to specify the number of pages to return from a given
topic which resulted in the number of returned items variing between one
and however many entries are listed on one page.

This commit changes the implementation for the limit to keep loading more
pages until the specified limit is reached. Excessive elements are removed
in order to return the exact amount of items specified by the limit.

This behavior is closer to how other bridges are implemented and makes it
more natural to use without being too confusing. Existing queries must be
updated to account for the new limit.

References #657
2018-04-06 22:25:49 +02:00
sysadminstory
c899399569 [DealabsBridge] Follow the website changes (#660) 2018-04-06 21:25:41 +02:00
LogMANOriginal
0f93370e92 Merge pull request #654 from LogMANOriginal/cURL
Use cURL instead of file_get_contents
2018-04-06 20:49:58 +02:00
logmanoriginal
45c3dcb636 [VkBridge] Simplify header specification 2018-04-06 20:42:19 +02:00
logmanoriginal
ecfc220b10 [KernelBugTrackerBridge] Fix too many parameters requesting HTML DOM 2018-04-06 20:42:19 +02:00
logmanoriginal
4b3efed7ec [YoutubeBridge] Fix too many parameters when using HTML mode 2018-04-06 20:42:19 +02:00
logmanoriginal
bc28c5da8e [contents] Set CURLOPT_HTTPHEADER only if the provided array contains data 2018-04-06 20:42:19 +02:00
logmanoriginal
5bd9c1611d [contents] Limit cURL protocols to HTTP and HTTPS 2018-04-06 20:42:19 +02:00
logmanoriginal
6caca4946b bridges: Fix bridges with custom headers and options
This commit fixes bridges which called getContents, getSimpleHTMLDOM
or getSimpleHTMLDOMCached with custom settings.
2018-04-06 20:42:19 +02:00
logmanoriginal
ee78e7613f [contents] Replace file_get_contents by cURL
cURL is a powerful library specifically designed to connect to many
different types of servers with different types of protocols. For
more detailed information refer to the PHP cURL manual:

- http://php.net/manual/en/book.curl.php

Due to this change some parameters for the getContents function were
necessary (also applies to getSimpleHTMLDOM and getSimpleHTMLDOMCached):

> $use_include_path removed

  This parameter has never been used and doesn't even make sense in
  this context; If set to true file_get_contents would also search
  for files in the include_path (specified in php.ini).

> $context replaced by $header and $opts

  The $context parameter allowed for customization of the request in
  order to change how file_get_contents would acquire the data (i.e.
  using POST instead of GET, sending custom header, etc...)

  cURL also provides facilities to specify custom headers and change
  how it communicates to severs. cURL, however, is much more advanced.

  - $header is an optional parameter (empty by default). It receives
    an array of strings to send in the HTTP request header.

    See 'CURLOPT_HTTPHEADER':

    "An array of HTTP header fields to set, in the format
    array('Content-type: text/plain', 'Content-length: 100')"

    - php.net/manual/en/function.curl-setopt.php

  - $opts is an optional parameter (empty by default). It receives
    an array of options, where each option is a key-value-pair of
    a cURL option (CURLOPT_*) and it's associated parameter. This
    parameter accepts any of the CURLOPT_* settings.

    Example (sending POST instead of GET):

    $opts = array(
      CURLOPT_POST => 1,
      CURLOPT_POSTFIELDS => '&action=none'
    );
    $html = getContents($url, array(), $opts);

    Refer to the cURL setopt manual for more information:
    - php.net/manual/en/function.curl-setopt.php

> $offset and $maxlen removed

  These options were supported by file_get_contents, but there doesn't
  seem to be an equivalent in cURL. Since no caller uses them they are
  safe to remove.

Compressed data / Encoding

  By using cURL instead of file_get_contents RSS-Bridge no longer has
  to handle compressed data manually.

  See 'CURLOPT_ENCODING':

  "[...] Supported encodings are "identity", "deflate", and "gzip".
  If an empty string, "", is set, a header containing all supported
  encoding types is sent."

  - http://php.net/manual/en/function.curl-setopt.php

  Notice: By default all encoding types are accepted (""). This can
  be changed by setting a custom option via $opts.

    Example:

    $opts = array(CURLOPT_ENCODING => 'gzip');
    $html = getContents($url, array(), $opts);

Proxy

The proxy implementation should still work, but there doesn't seem
to be an equivalent for 'request_fulluri = true'. To my understanding
this isn't an issue because cURL knows how to handle proxy communication.
2018-04-06 20:42:19 +02:00
logmanoriginal
2df2623430 [index] Add 'curl' extension check 2018-04-06 20:42:19 +02:00
logmanoriginal
de5f850cdb [index] Fix indentation using tabs 2018-04-06 20:34:44 +02:00
teromene
ac6847045c Catch Errors in order to display a message in more cases. We also catch Exceptions to maintain compat with php 5.
Add check for simplexml extension.
2018-04-04 19:02:40 +01:00
logmanoriginal
df6da837dc [FacebookBridge] Return error if username starts with slash
Requesting a username with a leading slash would cause error 500
because the requested URI would contain two slashes in a row.

For example username "/test" would result in:
https://facebook.com//test

References #628
2018-03-23 21:23:30 +01:00
Eugene Molotov
41b7984a4e [YoutubeBridge] Playlist mode: faster feed generating if item count is less or equal to 15 (#648)
* [YoutubeBridge] Playlist mode: faster feed generating if item count is less or equal to 15
2018-03-19 12:41:52 +00:00
teromene
38c7e0272e Add hashtag support to InstagramBridge.
Fixes  #629
2018-03-19 12:29:24 +00:00
teromene
29c690dbcd Fix InstagramBridge, thanks to @pintassilgo comments.
Fixes #646
2018-03-19 12:17:42 +00:00
LogMANOriginal
8ba817478b Implement customizable cache timeout (#641)
* [BridgeAbstract] Implement customizable cache timeout

The customizable cache timeout is used instead of the default cache
timeout (CACHE_TIMEOUT) if specified by the caller.

* [index] Add new global parameter '_cache_timeout'

The _cache_timeout parameter is an optional parameter that can be
used to specify a custom cache timeout. This option is enabled by
default.

It can be disabled using the named constant 'CUSTOM_CACHE_TIMEOUT'
which supports two states:

> true: Enabled (default)
> false: Disabled

* [BridgeAbstract] Change scope of 'getCacheTimeout' to public

* [html] Add cache timeout parameter to all bridges

The timeout parameter only shows if CUSTOM_CACHE_TIMEOUT has been set
to true. The default value is automatically set to the value specified
in the bridge.

* [index] Disable custom cache timeout by default
2018-03-14 18:06:36 +01:00
Eugene Molotov
cacbe90102 [YoutubeBridge] Sort playlist items by publication date (#643) 2018-03-13 11:24:40 +00:00
Antoine Cadoret
cb91cd5d2f Fix SteamBridge (#637) (#639)
Fixes #639
2018-03-12 09:22:34 +00:00
sysadminstory
52dfa3fe76 [RadioMelodieBridge] Add new bridge (#640) 2018-03-11 15:38:07 +01:00
778 changed files with 97692 additions and 16337 deletions

21
.devcontainer/Dockerfile Normal file
View File

@@ -0,0 +1,21 @@
FROM rssbridge/rss-bridge:latest
COPY --chmod=755 post-create-command.sh /usr/local/bin/post-create-command
ADD https://raw.githubusercontent.com/docker-library/php/master/docker-php-ext-enable /usr/local/bin/docker-php-ext-enable
RUN chmod u+x /usr/local/bin/docker-php-ext-enable
ADD https://getcomposer.org/installer /usr/local/bin/composer-installer.php
RUN chmod u+x /usr/local/bin/composer-installer.php
RUN php /usr/local/bin/composer-installer.php --check && \
php /usr/local/bin/composer-installer.php --filename=composer --install-dir=/usr/local/bin
RUN apt-get update && \
apt-get install -y \
git \
php-dev \
make \
unzip
RUN pecl install xdebug && \
PHP_INI_DIR=/etc/php/8.2/fpm docker-php-ext-enable xdebug

View File

@@ -0,0 +1,30 @@
{
"name": "rss-bridge dev",
"build": { "dockerfile": "Dockerfile" },
"customizations": {
// Configure properties specific to VS Code.
"vscode": {
// Set *default* container specific settings.json values on container create.
"settings": {
"php.validate.executablePath": "/usr/bin/php",
"phpSniffer.executablesFolder": "/root/.config/composer/vendor/bin",
"phpcs.executablePath": "/root/.config/composer/vendor/bin/phpcs",
"phpcs.lintOnType": false
},
// Add the IDs of extensions you want installed when the container is created.
"extensions": [
"xdebug.php-debug",
"bmewburn.vscode-intelephense-client",
"philfontaine.autolaunch",
"eamodio.gitlens",
"shevaua.phpcs"
]
}
},
"remoteEnv": {
"PATH": "${containerEnv:PATH}:/root/.config/composer/vendor/bin",
},
"forwardPorts": [3100, 9000, 9003],
"postCreateCommand": "/usr/local/bin/post-create-command"
}

50
.devcontainer/launch.json Normal file
View File

@@ -0,0 +1,50 @@
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Listen for Xdebug",
"type": "php",
"request": "launch",
"port": 9003,
"auto": true,
"log": true
},
{
"name": "Launch currently open script",
"type": "php",
"request": "launch",
"program": "${file}",
"cwd": "${fileDirname}",
"port": 0,
"runtimeArgs": [
"-dxdebug.start_with_request=yes"
],
"env": {
"XDEBUG_MODE": "debug,develop",
"XDEBUG_CONFIG": "client_port=${port}"
}
},
{
"name": "Launch Built-in web server",
"type": "php",
"request": "launch",
"runtimeArgs": [
"-dxdebug.mode=debug",
"-dxdebug.start_with_request=yes",
"-S",
"localhost:0"
],
"program": "",
"cwd": "${workspaceRoot}",
"port": 9003,
"serverReadyAction": {
"pattern": "Development Server \\(http://localhost:([0-9]+)\\) started",
"uriFormat": "http://localhost:%s",
"action": "openExternally"
}
}
]
}

17
.devcontainer/nginx.conf Normal file
View File

@@ -0,0 +1,17 @@
server {
listen 3100 default_server;
root /workspaces/rss-bridge;
access_log /var/log/nginx/rssbridge.access.log;
error_log /var/log/nginx/rssbridge.error.log;
index index.php;
location ~ /(\.|vendor|tests) {
deny all;
return 403; # Forbidden
}
location ~ \.php$ {
include snippets/fastcgi-php.conf;
fastcgi_pass unix:/var/run/php/php8.2-fpm.sock;
}
}

View File

@@ -0,0 +1,27 @@
#/bin/sh
cp .devcontainer/nginx.conf /etc/nginx/conf.d/default.conf
cp .devcontainer/xdebug.ini /etc/php/8.2/fpm/conf.d/xdebug.ini
# This should download some dev-dependencies, like phpunit and the PHP code sniffers
composer global require "phpunit/phpunit:^9"
composer global require "squizlabs/php_codesniffer:^3.6"
composer global require "phpcompatibility/php-compatibility:^9.3"
# We need to this manually for running the PHPCompatibility ruleset
phpcs --config-set installed_paths /root/.config/composer/vendor/phpcompatibility/php-compatibility
mkdir -p .vscode
cp .devcontainer/launch.json .vscode
echo '*' > whitelist.txt
chmod a+x $(pwd)
rm -rf /var/www/html
ln -s $(pwd) /var/www/html
# Solves possible issue of cache directory not being accessible
chown www-data:www-data -R $(pwd)/cache
nginx
php-fpm8.2 -D

7
.devcontainer/xdebug.ini Normal file
View File

@@ -0,0 +1,7 @@
[xdebug]
xdebug.mode=develop,debug
xdebug.client_host=localhost
xdebug.client_port=9003
xdebug.start_with_request=yes
xdebug.discover_client_host=false
xdebug.log='/var/www/html/xdebug.log'

16
.dockerignore Normal file
View File

@@ -0,0 +1,16 @@
.git
!.git/HEAD
!.git/refs/heads/*
.gitattributes
.github/*
.travis.yml
cache/*
CONTRIBUTING.md
DEBUG
Dockerfile
phpcompatibility.xml
phpcs.xml
phpcs.xml
scalingo.json
tests/*
whitelist.txt

4
.git-blame-ignore-revs Normal file
View File

@@ -0,0 +1,4 @@
# Reformat code base to PSR12
4f75591060d95208a301bc6bf460d875631b29cc
# Fix coding style missed by phpbcf
951092eef374db048b77bac85e75e3547bfac702

66
.gitattributes vendored
View File

@@ -1,5 +1,6 @@
# Auto detect text files and perform LF normalization
* text=auto
*.sh text eol=lf
# Custom for Visual Studio
*.cs diff=csharp
@@ -10,13 +11,58 @@
*.dbproj merge=union
# Standard to msysgit
*.doc diff=astextplain
*.DOC diff=astextplain
*.docx diff=astextplain
*.DOCX diff=astextplain
*.dot diff=astextplain
*.DOT diff=astextplain
*.pdf diff=astextplain
*.PDF diff=astextplain
*.rtf diff=astextplain
*.RTF diff=astextplain
*.doc diff=astextplain
*.DOC diff=astextplain
*.docx diff=astextplain
*.DOCX diff=astextplain
*.dot diff=astextplain
*.DOT diff=astextplain
*.pdf diff=astextplain
*.PDF diff=astextplain
*.rtf diff=astextplain
*.RTF diff=astextplain
# Ignore files in git archive (i.e. GitHub release builds)
## Docker
Dockerfile export-ignore
.dockerignore export-ignore
## Travis
.travis.yml export-ignore
## GitHub
.github/ export-ignore
## Git
.gitattributes export-ignore
.gitignore export-ignore
## Scalingo
scalingo.json export-ignore
## RSS-Bridge
phpunit.xml export-ignore
phpcs.xml export-ignore
phpcompatibility.xml export-ignore
tests/ export-ignore
cache/.gitkeep export-ignore
## Composer
#
# Keep the following lines commented out. Heroku does
# not function if the composer files are ignored during
# export. For more information see
# https://github.com/rss-bridge/rss-bridge/issues/1165
#
# composer.json export-ignore
# composer.lock export-ignore
## Heroku
#
# Keep the following line commented out. Heroku does
# not function if app.json is ignored during export.
# For more information see
# https://github.com/rss-bridge/rss-bridge/issues/1165
#
# app.json export-ignore

7
.github/.gitignore vendored Normal file
View File

@@ -0,0 +1,7 @@
# Visual Studio Code
.vscode/*
# Generated files
comment*.md
comment*.txt
*.html

7
.github/CONTRIBUTING.md vendored Normal file
View File

@@ -0,0 +1,7 @@
### Pull request policy
See the [Pull request policy page on the documentation](https://rss-bridge.github.io/rss-bridge/For_Developers/Pull_Request_policy.html) for more information on the pull request policy.
### Coding style
See the [Coding style policy page on the documentation](https://rss-bridge.github.io/rss-bridge/For_Developers/Coding_style_policy.html) for more information on the coding style of the project.

View File

@@ -0,0 +1,64 @@
---
name: Bridge request
about: Use this template for requesting a new bridge
title: Bridge request for ...
labels: Bridge-Request
assignees: ''
---
# Bridge request
<!--
This is a bridge request. Start by adding a descriptive title (i.e. `Bridge request for GitHub`). Use the "Preview" button to see a preview of your request. Make sure your request is complete before submitting!
Notice: This comment is only visible to you while you work on your request. Please do not remove any of the lines in the template (you may add your own outside the "<!--" and "- ->" lines!)
-->
## General information
<!--
Please describe what you expect from the bridge. Whenever possible provide sample links and screenshots (you can just paste them here) to express your expectations and help others understand your request. If possible, mark relevant areas in your screenshot. Use the following questions for reference:
-->
- _Host URI for the bridge_ (i.e. `https://github.com`):
- Which information would you like to see?
- How should the information be displayed/formatted?
- Which of the following parameters do you expect?
- [X] Title
- [X] URI (link to the original article)
- [ ] Author
- [ ] Timestamp
- [X] Content (the content of the article)
- [ ] Enclosures (pictures, videos, etc...)
- [ ] Categories (categories, tags, etc...)
## Options
<!--Select options from the list below. Add your own option if one is missing:-->
- [ ] Limit number of returned items
- _Default limit_: 5
- [ ] Load full articles
- _Cache articles_ (articles are stored in a local cache on first request): yes
- _Cache timeout_ : 24 hours
- [X] Balance requests (RSS-Bridge uses cached versions to reduce bandwith usage)
- _Timeout_ (default = 5 minutes): 5 minutes
<!--Be aware that some options might not be available for your specific request due to technical limitations!-->
<!--
## Additional notes
Keep in mind that opening a request does not guarantee the bridge being implemented! That depends entirely on the interest and time of others to make the bridge for you.
You can also implement your own bridge (with support of the community if needed). Find more information in the [RSS-Bridge Documentation](https://rss-bridge.github.io/rss-bridge/For_Developers/index.html) developer section.
-->

38
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@@ -0,0 +1,38 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: Bug-Report
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
**Smartphone (please complete the following information):**
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
**Additional context**
Add any other context about the problem here.

View File

@@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: Feature-Request
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.

2
.github/prtester-requirements.txt vendored Normal file
View File

@@ -0,0 +1,2 @@
beautifulsoup4>=4.10.0
requests>=2.26.0

208
.github/prtester.py vendored Normal file
View File

@@ -0,0 +1,208 @@
import argparse
import requests
import re
from bs4 import BeautifulSoup
from datetime import datetime
from typing import Iterable
import os
import glob
import urllib
# This script is specifically written to be used in automation for https://github.com/RSS-Bridge/rss-bridge
#
# This will scrape the whitelisted bridges in the current state (port 3000) and the PR state (port 3001) of
# RSS-Bridge, generate a feed for each of the bridges and save the output as html files.
# It also add a <base> tag with the url of em's public instance, so viewing
# the HTML file locally will actually work as designed.
ARTIFACT_FILE_EXTENSION = '.html'
class Instance:
name = ''
url = ''
def main(instances: Iterable[Instance], with_upload: bool, with_reduced_upload: bool, title: str, output_file: str):
start_date = datetime.now()
prid = os.getenv('PR')
artifact_base_url = f'https://rss-bridge.github.io/rss-bridge-tests/prs/{prid}'
artifact_directory = os.getcwd()
for file in glob.glob(f'*{ARTIFACT_FILE_EXTENSION}', root_dir=artifact_directory):
os.remove(file)
table_rows = []
for instance in instances:
page = requests.get(instance.url) # Use python requests to grab the rss-bridge main page
soup = BeautifulSoup(page.content, "html.parser") # use bs4 to turn the page into soup
bridge_cards = soup.select('.bridge-card') # get a soup-formatted list of all bridges on the rss-bridge page
table_rows += testBridges(
instance=instance,
bridge_cards=bridge_cards,
with_upload=with_upload,
with_reduced_upload=with_reduced_upload,
artifact_directory=artifact_directory,
artifact_base_url=artifact_base_url) # run the main scraping code with the list of bridges
with open(file=output_file, mode='w+', encoding='utf-8') as file:
table_rows_value = '\n'.join(sorted(table_rows))
file.write(f'''
## {title}
| Bridge | Context | Status |
| - | - | - |
{table_rows_value}
*last change: {start_date.strftime("%A %Y-%m-%d %H:%M:%S")}*
'''.strip())
def testBridges(instance: Instance, bridge_cards: Iterable, with_upload: bool, with_reduced_upload: bool, artifact_directory: str, artifact_base_url: str) -> Iterable:
instance_suffix = ''
if instance.name:
instance_suffix = f' ({instance.name})'
table_rows = []
for bridge_card in bridge_cards:
bridgeid = bridge_card.get('id')
bridgeid = bridgeid.split('-')[1] # this extracts a readable bridge name from the bridge metadata
print(f'{bridgeid}{instance_suffix}')
bridge_name = bridgeid.replace('Bridge', '')
context_forms = bridge_card.find_all("form")
form_number = 1
for context_form in context_forms:
# a bridge can have multiple contexts, named 'forms' in html
# this code will produce a fully working url that should create a working feed when called
# this will create an example feed for every single context, to test them all
context_parameters = {}
error_messages = []
context_name = '*untitled*'
context_name_element = context_form.find_previous_sibling('h5')
if context_name_element and context_name_element.text.strip() != '':
context_name = context_name_element.text
parameters = context_form.find_all("input")
lists = context_form.find_all("select")
# this for/if mess cycles through all available input parameters, checks if it required, then pulls
# the default or examplevalue and then combines it all together into the url parameters
# if an example or default value is missing for a required attribute, it will throw an error
# any non-required fields are not tested!!!
for parameter in parameters:
parameter_type = parameter.get('type')
parameter_name = parameter.get('name')
if parameter_type == 'hidden':
context_parameters[parameter_name] = parameter.get('value')
if parameter_type == 'number' or parameter_type == 'text':
if parameter.has_attr('required'):
if parameter.get('placeholder') == '':
if parameter.get('value') == '':
error_messages.append(f'Missing example or default value for parameter "{parameter_name}"')
else:
context_parameters[parameter_name] = parameter.get('value')
else:
context_parameters[parameter_name] = parameter.get('placeholder')
# same thing, just for checkboxes. If a checkbox is checked per default, it gets added to the url parameters
if parameter_type == 'checkbox':
if parameter.has_attr('checked'):
context_parameters[parameter_name] = 'on'
for listing in lists:
selectionvalue = ''
listname = listing.get('name')
cleanlist = []
options = listing.find_all('option')
for option in options:
if 'optgroup' in option.name:
cleanlist.extend(option)
else:
cleanlist.append(option)
firstselectionentry = 1
for selectionentry in cleanlist:
if firstselectionentry:
selectionvalue = selectionentry.get('value')
firstselectionentry = 0
else:
if 'selected' in selectionentry.attrs:
selectionvalue = selectionentry.get('value')
break
context_parameters[listname] = selectionvalue
artifact_url = 'about:blank'
if error_messages:
status = '<br>'.join(map(lambda m: f'❌ `{m}`', error_messages))
else:
# if all example/default values are present, form the full request url, run the request, add a <base> tag with
# the url of em's public instance to the response text (so that relative paths work, e.g. to the static css file) and
# then save it to a html file.
context_parameters.update({
'action': 'display',
'bridge': bridgeid,
'format': 'Html',
})
request_url = f'{instance.url}/?{urllib.parse.urlencode(context_parameters)}'
response = requests.get(request_url)
page_text = response.text.replace('<head>','<head><base href="https://rss-bridge.org/bridge01/" target="_blank">')
page_text = page_text.encode("utf_8")
soup = BeautifulSoup(page_text, "html.parser")
status_messages = []
if response.status_code != 200:
status_messages += [f'❌ `HTTP status {response.status_code} {response.reason}`']
else:
feed_items = soup.select('.feeditem')
feed_items_length = len(feed_items)
if feed_items_length <= 0:
status_messages += [f'⚠️ `The feed has no items`']
elif feed_items_length == 1 and len(soup.select('.error')) > 0:
status_messages += [f'❌ `{getFirstLine(feed_items[0].text)}`']
status_messages += map(lambda e: f'❌ `{getFirstLine(e.text)}`', soup.select('.error .error-type') + soup.select('.error .error-message'))
for item_element in soup.select('.feeditem'): # remove all feed items to not accidentally selected <pre> tags from item content
item_element.decompose()
status_messages += map(lambda e: f'⚠️ `{getFirstLine(e.text)}`', soup.find_all('pre'))
status_messages = list(dict.fromkeys(status_messages)) # remove duplicates
status = '<br>'.join(status_messages)
status_is_ok = status == '';
if status_is_ok:
status = '✔️'
if with_upload and (not with_reduced_upload or not status_is_ok):
filename = f'{bridge_name} {form_number}{instance_suffix}{ARTIFACT_FILE_EXTENSION}'
filename = re.sub(r'[^a-z0-9 \_\-\.]', '', filename, flags=re.I).replace(' ', '_')
with open(file=f'{artifact_directory}/{filename}', mode='wb') as file:
file.write(page_text)
artifact_url = f'{artifact_base_url}/{filename}'
table_rows.append(f'| {bridge_name} | [{form_number} {context_name}{instance_suffix}]({artifact_url}) | {status} |')
form_number += 1
return table_rows
def getFirstLine(value: str) -> str:
# trim whitespace and remove text that can break the table or is simply unnecessary
clean_value = re.sub(r'^\[[^\]]+\]\s*rssbridge\.|[\|`]', '', value.strip())
first_line = next(iter(clean_value.splitlines()), '')
max_length = 250
if (len(first_line) > max_length):
first_line = first_line[:max_length] + '...'
return first_line
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--instances', nargs='+')
parser.add_argument('--no-upload', action='store_true')
parser.add_argument('--reduced-upload', action='store_true')
parser.add_argument('--title', default='Pull request artifacts')
parser.add_argument('--output-file', default=os.getcwd() + '/comment.txt')
args = parser.parse_args()
instances = []
if args.instances:
for instance_arg in args.instances:
instance_arg_parts = instance_arg.split('::')
instance = Instance()
instance.name = instance_arg_parts[1].strip() if len(instance_arg_parts) >= 2 else ''
instance.url = instance_arg_parts[0].strip().rstrip("/")
instances.append(instance)
else:
instance = Instance()
instance.name = 'current'
instance.url = 'http://localhost:3000'
instances.append(instance)
instance = Instance()
instance.name = 'pr'
instance.url = 'http://localhost:3001'
instances.append(instance)
main(
instances=instances,
with_upload=not args.no_upload,
with_reduced_upload=args.reduced_upload and not args.no_upload,
title=args.title,
output_file=args.output_file
);

61
.github/workflows/dockerbuild.yml vendored Normal file
View File

@@ -0,0 +1,61 @@
name: Build Image on Commit and Release
on:
push:
branches:
- 'master'
tags:
- '20*'
env:
DOCKERHUB_SLUG: rssbridge/rss-bridge
GHCR_SLUG: ghcr.io/rss-bridge/rss-bridge
jobs:
bake:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Docker meta
id: docker_meta
uses: docker/metadata-action@v5
with:
images: |
${{ env.DOCKERHUB_SLUG }}
${{ env.GHCR_SLUG }}
tags: |
type=raw,value=latest
type=sha
type=ref,event=tag,enable=${{ startsWith(github.ref, 'refs/tags/20') }}
type=raw,value=stable,enable=${{ startsWith(github.ref, 'refs/tags/20') }}
-
name: Set up QEMU
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
-
name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
-
name: Build and push
uses: docker/bake-action@v5
with:
files: |
./docker-bake.hcl
${{ steps.docker_meta.outputs.bake-file }}
targets: image-all
push: true

27
.github/workflows/documentation.yml vendored Normal file
View File

@@ -0,0 +1,27 @@
name: Documentation
on:
push:
paths:
- 'docs/**'
jobs:
documentation:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: 8.0
- name: Install dependencies
run: composer global require daux/daux.io
- name: Generate documentation
run: daux generate
- name: Deploy same repository 🚀
uses: JamesIves/github-pages-deploy-action@v4
with:
folder: "static"
branch: gh-pages

49
.github/workflows/lint.yml vendored Normal file
View File

@@ -0,0 +1,49 @@
name: Lint
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
phpcs:
runs-on: ubuntu-22.04
strategy:
matrix:
php-versions: ['7.4']
steps:
- uses: actions/checkout@v4
- uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php-versions }}
tools: phpcs
- run: phpcs . --standard=phpcs.xml --warning-severity=0 --extensions=php -p
phpcompatibility:
runs-on: ubuntu-22.04
strategy:
matrix:
php-versions: ['7.4']
steps:
- uses: actions/checkout@v4
- uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php-versions }}
- run: composer global config --no-plugins allow-plugins.dealerdirect/phpcodesniffer-composer-installer true
- run: composer global require dealerdirect/phpcodesniffer-composer-installer
- run: composer global require phpcompatibility/php-compatibility
- run: ~/.composer/vendor/bin/phpcs . --standard=phpcompatibility.xml --warning-severity=0 --extensions=php -p
executable_php_files_check:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- run: |
if find -name "*.php" -executable -type f -print -exec false {} +
then
echo 'Good, no executable php scripts found'
else
echo 'Please unmark php scripts above as non-executable'
exit 1
fi

126
.github/workflows/prhtmlgenerator.yml vendored Normal file
View File

@@ -0,0 +1,126 @@
name: 'PR Testing'
on:
pull_request_target:
branches: [ master ]
jobs:
check-bridges:
name: Check if bridges were changed
runs-on: ubuntu-latest
outputs:
BRIDGES: ${{ steps.check1.outputs.BRIDGES }}
steps:
- name: Check number of bridges
id: check1
run: |
PR=${{github.event.number}};
wget https://patch-diff.githubusercontent.com/raw/$GITHUB_REPOSITORY/pull/$PR.patch;
bridgeamount=$(cat $PR.patch | grep "\bbridges/[A-Za-z0-9]*Bridge\.php\b" | sed "s=.*\bbridges/\([A-Za-z0-9]*\)Bridge\.php\b.*=\1=g" | sort | uniq | wc -l);
echo "BRIDGES=$bridgeamount" >> "$GITHUB_OUTPUT"
test-pr:
name: Generate HTML
runs-on: ubuntu-latest
needs: check-bridges
if: needs.check-bridges.outputs.BRIDGES > 0
env:
PYTHONUNBUFFERED: 1
# Needs additional permissions https://github.com/actions/first-interaction/issues/10#issuecomment-1041402989
steps:
- name: Check out self
uses: actions/checkout@v4
with:
ref: ${{github.event.pull_request.head.ref}}
repository: ${{github.event.pull_request.head.repo.full_name}}
- name: Check out rss-bridge
run: |
PR=${{github.event.number}};
wget -O requirements.txt https://raw.githubusercontent.com/$GITHUB_REPOSITORY/${{ github.event.pull_request.base.ref }}/.github/prtester-requirements.txt;
wget https://raw.githubusercontent.com/$GITHUB_REPOSITORY/${{ github.event.pull_request.base.ref }}/.github/prtester.py;
wget https://patch-diff.githubusercontent.com/raw/$GITHUB_REPOSITORY/pull/$PR.patch;
touch DEBUG;
cat $PR.patch | grep "\bbridges/[A-Za-z0-9]*Bridge\.php\b" | sed "s=.*\bbridges/\([A-Za-z0-9]*\)Bridge\.php\b.*=\1=g" | sort | uniq > whitelist.txt
- name: Start Docker - Current
run: |
docker run -d -v $GITHUB_WORKSPACE/whitelist.txt:/app/whitelist.txt -v $GITHUB_WORKSPACE/DEBUG:/app/DEBUG -p 3000:80 ghcr.io/rss-bridge/rss-bridge:latest
- name: Start Docker - PR
run: |
docker build -t prbuild .;
docker run -d -v $GITHUB_WORKSPACE/whitelist.txt:/app/whitelist.txt -v $GITHUB_WORKSPACE/DEBUG:/app/DEBUG -p 3001:80 prbuild
- name: Setup python
uses: actions/setup-python@v5
with:
python-version: '3.13'
cache: 'pip'
- name: Install requirements
run: |
cd $GITHUB_WORKSPACE
pip install -r requirements.txt
- name: Run bridge tests
id: testrun
run: |
mkdir results;
python prtester.py;
body="$(cat comment.txt)";
body="${body//'%'/'%25'}";
body="${body//$'\n'/'%0A'}";
body="${body//$'\r'/'%0D'}";
echo "bodylength=${#body}" >> $GITHUB_OUTPUT
env:
PR: ${{ github.event.number }}
- name: Upload generated tests
uses: actions/upload-artifact@v4
id: upload-generated-tests
with:
name: tests
path: '*.html'
- name: Find Comment
if: ${{ steps.testrun.outputs.bodylength > 130 }}
uses: peter-evans/find-comment@v3
id: fc
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: 'github-actions[bot]'
body-includes: Pull request artifacts
- name: Create or update comment
if: ${{ steps.testrun.outputs.bodylength > 130 }}
uses: peter-evans/create-or-update-comment@v4
with:
comment-id: ${{ steps.fc.outputs.comment-id }}
issue-number: ${{ github.event.pull_request.number }}
body-file: comment.txt
edit-mode: replace
upload_tests:
name: Upload tests
runs-on: ubuntu-latest
needs: test-pr
steps:
- uses: actions/checkout@v4
with:
repository: 'RSS-Bridge/rss-bridge-tests'
ref: 'main'
token: ${{ secrets.RSSTESTER_ACTION }}
- name: Setup git config
run: |
git config --global user.name "GitHub Actions"
git config --global user.email "<>"
- name: Download tests
uses: actions/download-artifact@v4
with:
name: tests
- name: Move tests
run: |
cd prs
mkdir -p ${{github.event.number}}
cd ${{github.event.number}}
mv -f $GITHUB_WORKSPACE/*.html .
- name: Commit and push generated tests
run: |
export COMMIT_MESSAGE="Added tests for PR ${{github.event.number}}"
git add .
git commit -m "$COMMIT_MESSAGE" || exit 0
git push

23
.github/workflows/tests.yml vendored Normal file
View File

@@ -0,0 +1,23 @@
name: Tests
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
phpunit8:
runs-on: ubuntu-22.04
strategy:
matrix:
php-versions: ['7.4', '8.0', '8.1', '8.2', '8.3', '8.4']
steps:
- uses: actions/checkout@v4
- uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php-versions }}
env:
update: true
- run: composer install
- run: composer test

19
.gitignore vendored
View File

@@ -6,7 +6,6 @@ data/
*.pydevproject
.project
.metadata
bin/
tmp/
*.tmp
*.bak
@@ -213,6 +212,7 @@ pip-log.txt
# Unit test / coverage reports
.coverage
.phpunit.result.cache
.tox
#Translations
@@ -227,8 +227,23 @@ pip-log.txt
/cache
/whitelist.txt
DEBUG
config.ini.php
config/*
!config/nginx.conf
!config/php-fpm.conf
!config/php.ini
######################
## VisualStudioCode ##
######################
.vscode/*
.vscode/*
#Builder
.buildconfig
#Auth
.htaccess
.htpasswd
#Crawler
robots.txt

View File

@@ -1,24 +0,0 @@
dist: trusty
sudo: false
language: php
install:
- pear channel-update pear.php.net
- pear install PHP_CodeSniffer
script:
- phpenv rehash
- phpcs . --standard=phpcs.xml --warning-severity=0 --extensions=php -p
matrix:
fast_finish: true
include:
- php: 5.6
- php: 7.0
- php: hhvm
- php: nightly
allow_failures:
- php: hhvm
- php: nightly

View File

@@ -1,263 +0,0 @@
rss-bridge Changelog
===
RSS-Bridge 2017-08-19
==
## General changes
* whitelist: Do case-insensitive whitelist matching
* [FeedExpander] Fix Serialization of 'SimpleXMLElement' is not allowed
* [FeedExpander] Remove whitespace from source content
* [index] Add GET parameter 'q' for search queries
- **Example**: You can now add `&q=Twitter` to load into the search field
* [index] Check permissions for cache folder and whitelist file
* [index] Show bridge options when loading with URL fragment
- **Example**: You can now add `#bridge-Twitter` to load the card with all
parameters visible
* [style] Center search cursor and hide placeholder
* [validation] Fix error on undefined optional numeric value
## Modified bridges
* [DanbooruBridge] Allow descendant classes to override tag collection
* [DribbbleBridge] Add dribble bridge listing last dribble popular shots (#558)
* [FacebookBridge] Fix &amp; in URLs
* [GelbooruBridge] Fix bridge not getting tags correctly
* [GoComicsBridge] Fix for page structure changes (#568)
* [LeBonCoinBridge] Fix bridge is marked executable
* [LWNprevBridge] Fix everchanging url
* [YoutubeBridge] Fix error on certain keywords
* [YoutubeBridge] Fix issues loading playlists
## Removed bridges
* VineBridge
RSS-Bridge 2017-08-03
==
## Important changes
* RSS-Bridge now has [contribution guidelines](CONTRIBUTING.md)
* [phpcs rules](phpcs.xml) follow the [contribution guidelines](CONTRIBUTING.md)
## General changes
* Added a search bar to make searching for bridges easier
* Added user friendly error page for when a bridge fails
* Added caching of extraInfos (name, uri)
* Added an indicator to warn for bridges using HTTP instead of HTTPS
* Various bug fixes and improvements
## Modified bridges
* AllocineFRBridge] Update Faux Raccord link
* [DanbooruBridge] Fix broken URI
* [DuckDuckGoBridge] Disable DuckDuckGo redirects so that the links returned are correct.
* [FacebookBridge] Add option to hide posts with facebook videos
* [FacebookBridge] Add requester languages to HTTP header
* [FacebookBridge] Handle summary posts
* [FacebookBridge] Replace 'novideo' with 'media_type'
* [FilterBridge] Initial implementation of basic title permit and block
* [FlickrTagBridge] Fix and improve bridge by using the FlickrExploreBridge approach
* [GooglePlusPostBridge] Autofix user names
* [GooglePlusPostBridge] Fix bridge implementation
* [GooglePlusPostBridge] Fix content loading
* [InstagramBridge] Add option to filter for videos and pictures
* [LWNprevBridge] full rewrite
* [MangareaderBridge] Fix double forward slashes
* [NasaApodBridge] Use HTTPS instead of HTTP
* [PinterestBridge] Fix checkbox not working
* [PinterestBridge] Fix implementation after DOM changes
* [RTBFBridge] Update URI
* [SexactuBridge] Fix URI and timestamp
* [SexactuBridge] Use most modern version of bridge api and cached pages (#504)
* [ShanaprojectBridge] Don't throw error if timestamp is missing
* [TwitterBridge] Add option to hide retweets
* [TwitterBridge] Avoid empty content caused by new login policy
* [TwitterBridge] Fix double slashes in URI
* [TwitterBridge] Fix missing spaces
* [TwitterBridge] Fix title includes anchors in plaintext format
* [TwitterBridge] ignore promoted tweets
* [TwitterBridge] Optimize returned image sizes
* [TwitterBridge] Show quotes and pictures
* [WebfailBridge] Properly handle gifs (DOM changed)
* [YoutubeBridge] Improve readability of feed contents
* [YoutubeBridge] Improve URL handling in video descriptions
## New bridges
* AmazonBridge
* DiceBridge
* EtsyBridge
* FB2Bridge
* FilterBridge
* FlickrBridge
* GithubSearchBridge
* GoComicsBridge
* KATBridge
* KernelBugTrackerBridge
* MixCloudBridge
* MoinMoinBridge
* RainbowSixSiegeBridge
* SteamBridge
* TheTVDBBridge
* Torrent9Bridge
* UsbekEtRicaBridge
* WikiLeaksBridge
* WordPressPluginUpdateBridge
Alpha 0.2
===
## Important changes
* RSS-Bridge has been [UNLICENSED](UNLICENSE)
* RSS-Bridge is now a community-managed project on [GitHub](https://github.com/rss-bridge/rss-bridge)
* RSS-Bridge now has a [Wiki](https://github.com/rss-bridge/rss-bridge/wiki)
* RSS-Bridge now supports [Travis-CI](https://travis-ci.org)
## General changes
* Added [CHANGELOG](CHANGELOG.md) (this file)
* Added [PHP Simple HTML DOM Parser](http://simplehtmldom.sourceforge.net) to [vendor](vendor/simplehtmldom/)
* Added cache purging function (cache will be force-purged after 24 hours or as defined by bridge)
* Added new format [MrssFormat](formats/MrssFormat.php)
* Added parameter `author` - for display of the feed author name - to all formats
* Added new abstraction of the BridgeInterface:
- [FeedExpander](https://github.com/RSS-Bridge/rss-bridge/wiki/Bridge-API)
* Added optional support for proxy usage on each individual bridge
* Added support for [custom bridge parameter](https://github.com/RSS-Bridge/rss-bridge/wiki/BridgeAbstract#format-specifications) (text, number, list, checkbox)
* Changed design of the welcome screen
* Changed design of HtmlFormat
* Changed behavior of debug mode:
- Enable debug mode by placing a file called "DEBUG" in the root folder
- Debug mode automatically disables cache file loading
* Changed implementation of bridges - see [Wiki](https://github.com/rss-bridge/rss-bridge/wiki)
- Changed comment-style metadata to constants
- Added support for multiple utilizations per bridge
- Changed the parameter loading algorithm to be loaded by RSS-Bridge core
* Improved checks for PHP version, configuration and extensions
* Many bug fixes
## Modified Bridges
* FlickrExploreBridge
* GoogleSearchBridge
* TwitterBridge
## New Bridges
* ABCTabsBridge
* AcrimedBridge
* AllocineFRBridge
* AnimeUltimeBridge
* Arte7Bridge
* AskfmBridge
* BandcampBridge
* BastaBridge
* BlaguesDeMerdeBridge
* BooruprojectBridge
* CADBridge
* CNETBridge
* CastorusBridge
* CollegeDeFranceBridge
* CommonDreamsBridge
* CopieDoubleBridge
* CourrierInternationalBridge
* CpasbienBridge
* CryptomeBridge
* DailymotionBridge
* DanbooruBridge
* DansTonChatBridge
* DauphineLibereBridge
* DemoBridge
* DeveloppezDotComBridge
* DilbertBridge
* DollbooruBridge
* DuckDuckGoBridge
* EZTVBridge
* EliteDangerousGalnetBridge
* ElsevierBridge
* EstCeQuonMetEnProdBridge
* FacebookBridge
* FierPandaBridge
* FlickrTagBridge
* FootitoBridge
* FourchanBridge
* FuturaSciencesBridge
* GBAtempBridge
* GelbooruBridge
* GiphyBridge
* GithubIssueBridge
* GizmodoBridge
* GooglePlusPostBridge
* HDWallpapersBridge
* HentaiHavenBridge
* IdenticaBridge
* InstagramBridge
* IsoHuntBridge
* JapanExpoBridge
* KonachanBridge
* KoreusBridge
* KununuBridge
* LWNprevBridge
* LeBonCoinBridge
* LegifranceJOBridge
* LeMondeInformatiqueBridge
* LesJoiesDuCodeBridge
* LichessBridge
* LinkedInCompanyBridge
* LolibooruBridge
* MangareaderBridge
* MilbooruBridge
* MoebooruBridge
* MondeDiploBridge
* MsnMondeBridge
* MspabooruBridge
* NasaApodBridge
* NeuviemeArtBridge
* NextInpactBridge
* NextgovBridge
* NiceMatinBridge
* NovelUpdatesBridge
* OpenClassroomsBridge
* ParuVenduImmoBridge
* PickyWallpapersBridge
* PinterestBridge
* PlanetLibreBridge
* RTBFBridge
* ReadComicsBridge
* Releases3DSBridge
* ReporterreBridge
* Rue89Bridge
* Rule34Bridge
* Rule34pahealBridge
* SafebooruBridge
* SakugabooruBridge
* ScmbBridge
* ScoopItBridge
* SensCritiqueBridge
* SexactuBridge
* ShanaprojectBridge
* Shimmie2Bridge
* SoundcloudBridge
* StripeAPIChangeLogBridge
* SuperbWallpapersBridge
* T411Bridge
* TagBoardBridge
* TbibBridge
* TheCodingLoveBridge
* TheHackerNewsBridge
* ThePirateBayBridge
* UnsplashBridge
* ViadeoCompanyBridge
* VineBridge
* VkBridge
* WallpaperStopBridge
* WebfailBridge
* WeLiveSecurityBridge
* WhydBridge
* WikipediaBridge
* WordPressBridge
* WorldOfTanksBridge
* XbooruBridge
* YandereBridge
* YoutubeBridge
* ZDNetBridge
Alpha 0.1
===
* First tagged version.
* Includes refactoring.
* Unstable.

View File

@@ -1,47 +0,0 @@
### Pull request policy
Fix one issue per pull request.
Squash commits before opening a pull request.
Respect the coding style policy.
Name your PR like the following :
* When correcting a single bridge, use `[BridgeName] Feature`.
* When fixing a problem in a specific file, use `[FileName] Feature`.
* When fixing a general problem, use `category : feature`.
Note that all pull-requests should pass the unit tests before they can be merged.
### Coding style
Use `camelCase` for variables and methods.
Use `UPPERCASE` for constants.
Use `PascalCase` for class names. When creating a bridge, your class and PHP file should be named `MyImplementationBridge`.
Use tabs for indentation.
Add an empty line at the end of your file.
Use `''` to encapsulate strings, including in arrays.
Prefer lines shorter than 80 chars, no line longer than 120 chars.
PHP constants should be in lower case (`true, false, null`...)
* Add spaces between the logical operator and your expressions (not needed for the `!` operator).
* Use `||` and `&&` instead of `or` and `and`.
* Add space between your condition and the opening bracket/closing bracket.
* Don't put a space between `if` and your bracket.
* Use `elseif` instead of `else if`.
* Add new lines in your conditions if they are containing more than one line.
* Example :
```PHP
if($a == true && $b) {
print($a);
} else if(!$b) {
$a = !$a;
$b = $b >> $a;
print($b);
} else {
print($b);
}
```

225
CONTRIBUTORS.md Normal file
View File

@@ -0,0 +1,225 @@
# Contributors
* [16mhz](https://github.com/16mhz)
* [adamchainz](https://github.com/adamchainz)
* [Ahiles3005](https://github.com/Ahiles3005)
* [akirk](https://github.com/akirk)
* [Albirew](https://github.com/Albirew)
* [aledeg](https://github.com/aledeg)
* [alex73](https://github.com/alex73)
* [alexAubin](https://github.com/alexAubin)
* [Alkarex](https://github.com/Alkarex)
* [AmauryCarrade](https://github.com/AmauryCarrade)
* [arnd-s](https://github.com/arnd-s)
* [ArthurHoaro](https://github.com/ArthurHoaro)
* [Astalaseven](https://github.com/Astalaseven)
* [Astyan-42](https://github.com/Astyan-42)
* [austinhuang0131](https://github.com/austinhuang0131)
* [axor-mst](https://github.com/axor-mst)
* [ayacoo](https://github.com/ayacoo)
* [az5he6ch](https://github.com/az5he6ch)
* [b1nj](https://github.com/b1nj)
* [benasse](https://github.com/benasse)
* [Binnette](https://github.com/Binnette)
* [BoboTiG](https://github.com/BoboTiG)
* [Bockiii](https://github.com/Bockiii)
* [brtsos](https://github.com/brtsos)
* [captn3m0](https://github.com/captn3m0)
* [chemel](https://github.com/chemel)
* [Chouchen](https://github.com/Chouchen)
* [ckiw](https://github.com/ckiw)
* [cn-tools](https://github.com/cn-tools)
* [cnlpete](https://github.com/cnlpete)
* [corenting](https://github.com/corenting)
* [couraudt](https://github.com/couraudt)
* [csisoap](https://github.com/csisoap)
* [da2x](https://github.com/da2x)
* [dabenzel](https://github.com/dabenzel)
* [Daiyousei](https://github.com/Daiyousei)
* [dawidsowa](https://github.com/dawidsowa)
* [DevonHess](https://github.com/DevonHess)
* [dhuschde](https://github.com/dhuschde)
* [disk0x](https://github.com/disk0x)
* [DJCrashdummy](https://github.com/DJCrashdummy)
* [Djuuu](https://github.com/Djuuu)
* [DnAp](https://github.com/DnAp)
* [dominik-th](https://github.com/dominik-th)
* [Draeli](https://github.com/Draeli)
* [Dreckiger-Dan](https://github.com/Dreckiger-Dan)
* [drego85](https://github.com/drego85)
* [drklee3](https://github.com/drklee3)
* [DRogueRonin](https://github.com/DRogueRonin)
* [dvikan](https://github.com/dvikan)
* [eggwhalefrog](https://github.com/eggwhalefrog)
* [em92](https://github.com/em92)
* [eMerzh](https://github.com/eMerzh)
* [EtienneM](https://github.com/EtienneM)
* [f0086](https://github.com/f0086)
* [fanch317](https://github.com/fanch317)
* [fatuuse](https://github.com/fatuuse)
* [fivefilters](https://github.com/fivefilters)
* [floviolleau](https://github.com/floviolleau)
* [fluffy-critter](https://github.com/fluffy-critter)
* [fmachen](https://github.com/fmachen)
* [Frenzie](https://github.com/Frenzie)
* [fulmeek](https://github.com/fulmeek)
* [ggiessen](https://github.com/ggiessen)
* [gileri](https://github.com/gileri)
* [Ginko-Aloe](https://github.com/Ginko-Aloe)
* [girlpunk](https://github.com/girlpunk)
* [Glandos](https://github.com/Glandos)
* [gloony](https://github.com/gloony)
* [GregThib](https://github.com/GregThib)
* [griffaurel](https://github.com/griffaurel)
* [Grummfy](https://github.com/Grummfy)
* [gsantner](https://github.com/gsantner)
* [guigot](https://github.com/guigot)
* [hollowleviathan](https://github.com/hollowleviathan)
* [hpacleb](https://github.com/hpacleb)
* [hunhejj](https://github.com/hunhejj)
* [husim0](https://github.com/husim0)
* [IceWreck](https://github.com/IceWreck)
* [imagoiq](https://github.com/imagoiq)
* [j0k3r](https://github.com/j0k3r)
* [JackNUMBER](https://github.com/JackNUMBER)
* [jacquesh](https://github.com/jacquesh)
* [jakubvalenta](https://github.com/jakubvalenta)
* [JasonGhent](https://github.com/JasonGhent)
* [jcgoette](https://github.com/jcgoette)
* [jdesgats](https://github.com/jdesgats)
* [jdigilio](https://github.com/jdigilio)
* [JeremyRand](https://github.com/JeremyRand)
* [JimDog546](https://github.com/JimDog546)
* [jNullj](https://github.com/jNullj)
* [Jocker666z](https://github.com/Jocker666z)
* [johnnygroovy](https://github.com/johnnygroovy)
* [johnpc](https://github.com/johnpc)
* [joni1993](https://github.com/joni1993)
* [jtojnar](https://github.com/jtojnar)
* [KamaleiZestri](https://github.com/KamaleiZestri)
* [kkoyung](https://github.com/kkoyung)
* [klimplant](https://github.com/klimplant)
* [KN4CK3R](https://github.com/KN4CK3R)
* [kolarcz](https://github.com/kolarcz)
* [kranack](https://github.com/kranack)
* [kraoc](https://github.com/kraoc)
* [krisu5](https://github.com/krisu5)
* [l1n](https://github.com/l1n)
* [laBecasse](https://github.com/laBecasse)
* [lagaisse](https://github.com/lagaisse)
* [lalannev](https://github.com/lalannev)
* [langfingaz](https://github.com/langfingaz)
* [lassana](https://github.com/lassana)
* [ldidry](https://github.com/ldidry)
* [Leomaradan](https://github.com/Leomaradan)
* [leyrer](https://github.com/leyrer)
* [liamka](https://github.com/liamka)
* [Limero](https://github.com/Limero)
* [LogMANOriginal](https://github.com/LogMANOriginal)
* [lorenzos](https://github.com/lorenzos)
* [lukasklinger](https://github.com/lukasklinger)
* [m0zes](https://github.com/m0zes)
* [Mar-Koeh](https://github.com/Mar-Koeh)
* [marcus-at-localhost](https://github.com/marcus-at-localhost)
* [marius8510000-bot](https://github.com/marius8510000-bot)
* [matthewseal](https://github.com/matthewseal)
* [mcbyte-it](https://github.com/mcbyte-it)
* [mdemoss](https://github.com/mdemoss)
* [melangue](https://github.com/melangue)
* [metaMMA](https://github.com/metaMMA)
* [mibe](https://github.com/mibe)
* [mickaelBert](https://github.com/mickaelBert)
* [mightymt](https://github.com/mightymt)
* [mitsukarenai](https://github.com/mitsukarenai)
* [Monocularity](https://github.com/Monocularity)
* [MonsieurPoutounours](https://github.com/MonsieurPoutounours)
* [mr-flibble](https://github.com/mr-flibble)
* [mro](https://github.com/mro)
* [mschwld](https://github.com/mschwld)
* [muekoeff](https://github.com/muekoeff)
* [mw80](https://github.com/mw80)
* [mxmehl](https://github.com/mxmehl)
* [Mynacol](https://github.com/Mynacol)
* [nel50n](https://github.com/nel50n)
* [niawag](https://github.com/niawag)
* [Niehztog](https://github.com/Niehztog)
* [NikNikYkt](https://github.com/NikNikYkt)
* [Nono-m0le](https://github.com/Nono-m0le)
* [NotsoanoNimus](https://github.com/NotsoanoNimus)
* [obsiwitch](https://github.com/obsiwitch)
* [Ololbu](https://github.com/Ololbu)
* [ORelio](https://github.com/ORelio)
* [otakuf](https://github.com/otakuf)
* [Park0](https://github.com/Park0)
* [Paroleen](https://github.com/Paroleen)
* [Patricol](https://github.com/Patricol)
* [paulchen](https://github.com/paulchen)
* [PaulVayssiere](https://github.com/PaulVayssiere)
* [pellaeon](https://github.com/pellaeon)
* [PeterDaveHello](https://github.com/PeterDaveHello)
* [Peterr-K](https://github.com/Peterr-K)
* [Piranhaplant](https://github.com/Piranhaplant)
* [pirnz](https://github.com/pirnz)
* [pit-fgfjiudghdf](https://github.com/pit-fgfjiudghdf)
* [pitchoule](https://github.com/pitchoule)
* [pmaziere](https://github.com/pmaziere)
* [Pofilo](https://github.com/Pofilo)
* [prysme01](https://github.com/prysme01)
* [pubak42](https://github.com/pubak42)
* [Qluxzz](https://github.com/Qluxzz)
* [quentinus95](https://github.com/quentinus95)
* [quickwick](https://github.com/quickwick)
* [rakoo](https://github.com/rakoo)
* [RawkBob](https://github.com/RawkBob)
* [regisenguehard](https://github.com/regisenguehard)
* [Riduidel](https://github.com/Riduidel)
* [rogerdc](https://github.com/rogerdc)
* [Roliga](https://github.com/Roliga)
* [ronansalmon](https://github.com/ronansalmon)
* [rremizov](https://github.com/rremizov)
* [s0lesurviv0r](https://github.com/s0lesurviv0r)
* [sal0max](https://github.com/sal0max)
* [sebsauvage](https://github.com/sebsauvage)
* [shutosg](https://github.com/shutosg)
* [simon816](https://github.com/simon816)
* [Simounet](https://github.com/Simounet)
* [somini](https://github.com/somini)
* [SpangleLabs](https://github.com/SpangleLabs)
* [SqrtMinusOne](https://github.com/SqrtMinusOne)
* [squeek502](https://github.com/squeek502)
* [StelFux](https://github.com/StelFux)
* [stjohnjohnson](https://github.com/stjohnjohnson)
* [Stopka](https://github.com/Stopka)
* [Strubbl](https://github.com/Strubbl)
* [sublimz](https://github.com/sublimz)
* [sunchaserinfo](https://github.com/sunchaserinfo)
* [SuperSandro2000](https://github.com/SuperSandro2000)
* [sysadminstory](https://github.com/sysadminstory)
* [t0stiman](https://github.com/t0stiman)
* [tameroski](https://github.com/tameroski)
* [teromene](https://github.com/teromene)
* [tgkenney](https://github.com/tgkenney)
* [thefranke](https://github.com/thefranke)
* [TheRadialActive](https://github.com/TheRadialActive)
* [theScrabi](https://github.com/theScrabi)
* [thezeroalpha](https://github.com/thezeroalpha)
* [thibaultcouraud](https://github.com/thibaultcouraud)
* [timendum](https://github.com/timendum)
* [TitiTestScalingo](https://github.com/TitiTestScalingo)
* [tomaszkane](https://github.com/tomaszkane)
* [tomershvueli](https://github.com/tomershvueli)
* [TotalCaesar659](https://github.com/TotalCaesar659)
* [tpikonen](https://github.com/tpikonen)
* [TReKiE](https://github.com/TReKiE)
* [triatic](https://github.com/triatic)
* [User123698745](https://github.com/User123698745)
* [VerifiedJoseph](https://github.com/VerifiedJoseph)
* [vitkabele](https://github.com/vitkabele)
* [WalterBarrett](https://github.com/WalterBarrett)
* [wtuuju](https://github.com/wtuuju)
* [xurxof](https://github.com/xurxof)
* [yamanq](https://github.com/yamanq)
* [yardenac](https://github.com/yardenac)
* [ymeister](https://github.com/ymeister)
* [yue-dongchen](https://github.com/yue-dongchen)
* [ZeNairolf](https://github.com/ZeNairolf)

75
Dockerfile Normal file
View File

@@ -0,0 +1,75 @@
FROM debian:12-slim AS rssbridge
LABEL description="RSS-Bridge is a PHP project capable of generating RSS and Atom feeds for websites that don't have one."
LABEL repository="https://github.com/RSS-Bridge/rss-bridge"
LABEL website="https://github.com/RSS-Bridge/rss-bridge"
ARG DEBIAN_FRONTEND=noninteractive
RUN set -xe && \
apt-get update && \
apt-get install --yes --no-install-recommends \
ca-certificates \
nginx \
nss-plugin-pem \
php-curl \
php-fpm \
php-intl \
# php-json is enabled by default with PHP 8.2 in Debian 12
php-mbstring \
php-memcached \
# php-opcache is enabled by default with PHP 8.2 in Debian 12
# php-openssl is enabled by default with PHP 8.2 in Debian 12
php-sqlite3 \
php-xml \
php-zip \
# php-zlib is enabled by default with PHP 8.2 in Debian 12
# for downloading libcurl-impersonate
curl \
# for patching libcurl-impersonate
patchelf \
&& \
# install curl-impersonate library
curlimpersonate_version=1.0.0rc2 && \
{ \
{ \
[ $(arch) = 'aarch64' ] && \
archive="libcurl-impersonate-v${curlimpersonate_version}.aarch64-linux-gnu.tar.gz" && \
sha512sum="c8add80e7a0430a074edea1a11f73d03044c48e848e164af2d6f362866623e29bede207a50f18f95b1bc5ab3d33f5c31408be60a6da66b74a0d176eebe299116" \
; } \
|| { \
[ $(arch) = 'armv7l' ] && \
archive="libcurl-impersonate-v${curlimpersonate_version}.arm-linux-gnueabihf.tar.gz" && \
sha512sum="d0403ca4ad55a8d499b120e5675c7b5a0dc4946af49c933e91fc24455ffe5e122aa21ee95554612ff5d1bd6faea1556e1e1b9c821918e2644cc9bcbddc05747a" \
; } \
|| { \
[ $(arch) = 'x86_64' ] && \
archive="libcurl-impersonate-v${curlimpersonate_version}.x86_64-linux-gnu.tar.gz" && \
sha512sum="35cafda2b96df3218a6d8545e0947a899837ede51c90f7ef2980bd2d99dbd67199bc620000df28b186727300b8c7046d506807fb48ee0fbc068dc8ae01986339" \
; } \
} && \
curl -LO "https://github.com/lexiforest/curl-impersonate/releases/download/v${curlimpersonate_version}/${archive}" && \
echo "$sha512sum $archive" | sha512sum -c - && \
mkdir -p /usr/local/lib/curl-impersonate && \
tar xaf "$archive" -C /usr/local/lib/curl-impersonate && \
patchelf --set-soname libcurl.so.4 /usr/local/lib/curl-impersonate/libcurl-impersonate.so && \
rm "$archive" && \
apt-get purge --assume-yes curl patchelf && \
rm -rf /var/lib/apt/lists/*
ENV LD_PRELOAD /usr/local/lib/curl-impersonate/libcurl-impersonate.so
ENV CURL_IMPERSONATE chrome131
# logs should go to stdout / stderr
RUN ln -sfT /dev/stderr /var/log/nginx/error.log; \
ln -sfT /dev/stdout /var/log/nginx/access.log; \
chown -R --no-dereference www-data:adm /var/log/nginx/
COPY ./config/nginx.conf /etc/nginx/sites-available/default
COPY ./config/php-fpm.conf /etc/php/8.2/fpm/pool.d/rss-bridge.conf
COPY ./config/php.ini /etc/php/8.2/fpm/conf.d/90-rss-bridge.ini
COPY --chown=www-data:www-data ./ /app/
EXPOSE 80
ENTRYPOINT ["/app/docker-entrypoint.sh"]

620
README.md
View File

@@ -1,142 +1,538 @@
rss-bridge
===
[![LICENSE](https://img.shields.io/badge/license-UNLICENSE-blue.svg)](UNLICENSE) [![Build Status](https://travis-ci.org/RSS-Bridge/rss-bridge.svg?branch=master)](https://travis-ci.org/RSS-Bridge/rss-bridge)
# RSS-Bridge
rss-bridge is a PHP project capable of generating ATOM feeds for websites which don't have one.
![RSS-Bridge](static/logo_600px.png)
Supported sites/pages (main)
===
RSS-Bridge is a PHP web application.
* `Bandcamp` : Returns last release from [bandcamp](https://bandcamp.com/) for a tag
* `Cryptome` : Returns the most recent documents from [Cryptome.org](http://cryptome.org/)
* `DansTonChat`: Most recent quotes from [danstonchat.com](http://danstonchat.com/)
* `DuckDuckGo`: Most recent results from [DuckDuckGo.com](https://duckduckgo.com/)
* `Facebook` : Returns the latest posts on a page or profile on [Facebook](https://facebook.com/)
* `FlickrExplore` : [Latest interesting images](http://www.flickr.com/explore) from Flickr
* `GooglePlus` : Most recent posts of user timeline
* `GoogleSearch` : Most recent results from Google Search
* `Identi.ca` : Identica user timeline (Should be compatible with other Pump.io instances)
* `Instagram`: Most recent photos from an Instagram user
* `OpenClassrooms`: Lastest tutorials from [fr.openclassrooms.com](http://fr.openclassrooms.com/)
* `Pinterest`: Most recent photos from user or search
* `ScmbBridge`: Newest stories from [secouchermoinsbete.fr](http://secouchermoinsbete.fr/)
* `ThePirateBay` : Returns the newest indexed torrents from [The Pirate Bay](https://thepiratebay.se/) with keywords
* `Twitter` : Return keyword/hashtag search or user timeline
* `Wikipedia`: highlighted articles from [Wikipedia](https://wikipedia.org/) in English, German, French or Esperanto
* `YouTube` : YouTube user channel, playlist or search
It generates web feeds for websites that don't have one.
Plus [many other bridges](bridges/) to enable, thanks to the community
Officially hosted instance: https://rss-bridge.org/bridge01/
Output format
===
Output format can take several forms:
IRC channel #rssbridge at https://libera.chat/
* `Atom` : ATOM Feed, for use in RSS/Feed readers
* `Html` : Simple html page.
* `Json` : Json, for consumption by other applications.
* `Mrss` : MRSS Feed, for use in RSS/Feed readers
* `Plaintext` : raw text (php object, as returned by print_r)
Screenshot
===
[Full documentation](https://rss-bridge.github.io/rss-bridge/index.html)
Welcome screen:
Alternatively find another
[public instance](https://rss-bridge.github.io/rss-bridge/General/Public_Hosts.html).
![Screenshot](https://github.com/RSS-Bridge/rss-bridge/wiki/images/screenshot_rss-bridge_welcome.png)
RSS-Bridge hashtag (#rss-bridge) search on Twitter, in ATOM format (as displayed by Firefox):
Requires minimum PHP 7.4.
![Screenshot](https://github.com/RSS-Bridge/rss-bridge/wiki/images/screenshot_twitterbridge_atom.png)
Requirements
===
* PHP 5.6, e.g. `AddHandler application/x-httpd-php56 .php` in `.htaccess`
* `openssl` extension enabled in PHP config (`php.ini`)
* `allow_url_fopen=1` in `php.ini`
[![LICENSE](https://img.shields.io/badge/license-UNLICENSE-blue.svg)](UNLICENSE)
[![GitHub release](https://img.shields.io/github/release/rss-bridge/rss-bridge.svg?logo=github)](https://github.com/rss-bridge/rss-bridge/releases/latest)
[![irc.libera.chat](https://img.shields.io/badge/irc.libera.chat-%23rssbridge-blue.svg)](https://web.libera.chat/#rssbridge)
[![Actions Status](https://img.shields.io/github/actions/workflow/status/RSS-Bridge/rss-bridge/tests.yml?branch=master&label=GitHub%20Actions&logo=github)](https://github.com/RSS-Bridge/rss-bridge/actions)
Enabling/Disabling bridges
===
|||
|:-:|:-:|
|![Screenshot #1](/static/screenshot-1.png?raw=true)|![Screenshot #2](/static/screenshot-2.png?raw=true)|
|![Screenshot #3](/static/screenshot-3.png?raw=true)|![Screenshot #4](/static/screenshot-4.png?raw=true)|
|![Screenshot #5](/static/screenshot-5.png?raw=true)|![Screenshot #6](/static/screenshot-6.png?raw=true)|
By default, the script creates `whitelist.txt` and adds the main bridges (see above). `whitelist.txt` is ignored by git, you can edit it:
* to enable extra bridges (one bridge per line)
* to disable main bridges (remove the line)
* to enable all bridges (just one wildcard `*` as file content)
## A subset of bridges (15/447)
New bridges are disabled by default, so make sure to check regularly what's new and whitelist what you want!
* `CssSelectorBridge`: [Scrape out a feed using CSS selectors](https://rss-bridge.org/bridge01/#bridge-CssSelectorBridge)
* `FeedMergeBridge`: [Combine multiple feeds into one](https://rss-bridge.org/bridge01/#bridge-FeedMergeBridge)
* `FeedReducerBridge`: [Reduce a noisy feed by some percentage](https://rss-bridge.org/bridge01/#bridge-FeedReducerBridge)
* `FilterBridge`: [Filter a feed by excluding/including items by keyword](https://rss-bridge.org/bridge01/#bridge-FilterBridge)
* `GettrBridge`: [Fetches the latest posts from a GETTR user](https://rss-bridge.org/bridge01/#bridge-GettrBridge)
* `MastodonBridge`: [Fetches statuses from a Mastodon (ActivityPub) instance](https://rss-bridge.org/bridge01/#bridge-MastodonBridge)
* `RedditBridge`: [Fetches posts from a user/subredit (with filtering options)](https://rss-bridge.org/bridge01/#bridge-RedditBridge)
* `RumbleBridge`: [Fetches channel/user videos](https://rss-bridge.org/bridge01/#bridge-RumbleBridge)
* `SoundcloudBridge`: [Fetches music by username](https://rss-bridge.org/bridge01/#bridge-SoundcloudBridge)
* `TelegramBridge`: [Fetches posts from a public channel](https://rss-bridge.org/bridge01/#bridge-TelegramBridge)
* `ThePirateBayBridge:` [Fetches torrents by search/user/category](https://rss-bridge.org/bridge01/#bridge-ThePirateBayBridge)
* `TikTokBridge`: [Fetches posts by username](https://rss-bridge.org/bridge01/#bridge-TikTokBridge)
* `TwitchBridge`: [Fetches videos from channel](https://rss-bridge.org/bridge01/#bridge-TwitchBridge)
* `XPathBridge`: [Scrape out a feed using XPath expressions](https://rss-bridge.org/bridge01/#bridge-XPathBridge)
* `YoutubeBridge`: [Fetches videos by username/channel/playlist/search](https://rss-bridge.org/bridge01/#bridge-YoutubeBridge)
* `YouTubeCommunityTabBridge`: [Fetches posts from a channel's Posts tab](https://rss-bridge.org/bridge01/#bridge-YouTubeCommunityTabBridge)
## Tutorial
### How to install on traditional shared web hosting
RSS-Bridge can basically be unzipped into a web folder. Should be working instantly.
Latest zip:
https://github.com/RSS-Bridge/rss-bridge/archive/refs/heads/master.zip (2MB)
### How to install on Debian 12 (nginx + php-fpm)
These instructions have been tested on a fresh Debian 12 VM from Digital Ocean (1vcpu-512mb-10gb, 5 USD/month).
```shell
timedatectl set-timezone Europe/Oslo
apt install git nginx php8.2-fpm php-mbstring php-simplexml php-curl php-intl
# Create a user account
useradd --shell /bin/bash --create-home rss-bridge
cd /var/www
# Create folder and change its ownership to rss-bridge
mkdir rss-bridge && chown rss-bridge:rss-bridge rss-bridge/
# Become rss-bridge
su rss-bridge
# Clone master branch into existing folder
git clone https://github.com/RSS-Bridge/rss-bridge.git rss-bridge/
cd rss-bridge
# Copy over the default config (OPTIONAL)
cp -v config.default.ini.php config.ini.php
# Recursively give full permissions to user/owner
chmod 700 --recursive ./
# Give read and execute to others on folder ./static
chmod o+rx ./ ./static
# Recursively give give read to others on folder ./static
chmod o+r --recursive ./static
```
Nginx config:
```nginx
# /etc/nginx/sites-enabled/rss-bridge.conf
server {
listen 80;
# TODO: change to your own server name
server_name example.com;
access_log /var/log/nginx/rss-bridge.access.log;
error_log /var/log/nginx/rss-bridge.error.log;
log_not_found off;
# Intentionally not setting a root folder
# Static content only served here
location /static/ {
alias /var/www/rss-bridge/static/;
}
# Pass off to php-fpm only when location is EXACTLY == /
location = / {
root /var/www/rss-bridge/;
include snippets/fastcgi-php.conf;
fastcgi_read_timeout 45s;
fastcgi_pass unix:/run/php/rss-bridge.sock;
}
# Reduce log noise
location = /favicon.ico {
access_log off;
}
# Reduce log noise
location = /robots.txt {
access_log off;
}
}
```
PHP FPM pool config:
```ini
; /etc/php/8.2/fpm/pool.d/rss-bridge.conf
[rss-bridge]
user = rss-bridge
group = rss-bridge
listen = /run/php/rss-bridge.sock
listen.owner = www-data
listen.group = www-data
; Create 10 workers standing by to serve requests
pm = static
pm.max_children = 10
; Respawn worker after 500 requests (workaround for memory leaks etc.)
pm.max_requests = 500
```
PHP ini config:
```ini
; /etc/php/8.2/fpm/conf.d/30-rss-bridge.ini
max_execution_time = 15
memory_limit = 64M
```
Restart fpm and nginx:
```shell
# Lint and restart php-fpm
php-fpm8.2 -t && systemctl restart php8.2-fpm
# Lint and restart nginx
nginx -t && systemctl restart nginx
```
### How to install from Composer
Install the latest release.
```shell
cd /var/www
composer create-project -v --no-dev --no-scripts rss-bridge/rss-bridge
```
### How to install with Caddy
TODO. See https://github.com/RSS-Bridge/rss-bridge/issues/3785
### Install from Docker Hub:
Install by downloading the docker image from Docker Hub:
```bash
# Create container
docker create --name=rss-bridge --publish 3000:80 --volume $(pwd)/config:/config rssbridge/rss-bridge
```
You can put custom `config.ini.php` and bridges into `./config`.
**You must restart container for custom changes to take effect.**
See `docker-entrypoint.sh` for details.
```bash
# Start container
docker start rss-bridge
```
Browse http://localhost:3000/
### Install by locally building from Dockerfile
```bash
# Build image from Dockerfile
docker build -t rss-bridge .
# Create container
docker create --name rss-bridge --publish 3000:80 --volume $(pwd)/config:/config rss-bridge
```
You can put custom `config.ini.php` and bridges into `./config`.
**You must restart container for custom changes to take effect.**
See `docker-entrypoint.sh` for details.
```bash
# Start container
docker start rss-bridge
```
Browse http://localhost:3000/
### Install with docker-compose (using Docker Hub)
You can put custom `config.ini.php` and bridges into `./config`.
**You must restart container for custom changes to take effect.**
See `docker-entrypoint.sh` for details.
```bash
docker-compose up
```
Browse http://localhost:3000/
### Other installation methods
Deploy
===
[![Deploy on Scalingo](https://cdn.scalingo.com/deploy/button.svg)](https://my.scalingo.com/deploy?source=https://github.com/sebsauvage/rss-bridge)
Authors
===
We are RSS Bridge Community, a group of developers continuing the project initiated by sebsauvage, webmaster of [sebsauvage.net](http://sebsauvage.net), author of [Shaarli](http://sebsauvage.net/wiki/doku.php?id=php:shaarli) and [ZeroBin](http://sebsauvage.net/wiki/doku.php?id=php:zerobin).
[![Deploy to Heroku](https://www.herokucdn.com/deploy/button.svg)](https://heroku.com/deploy)
[![Deploy to Cloudron](https://cloudron.io/img/button.svg)](https://www.cloudron.io/store/com.rssbridgeapp.cloudronapp.html)
[![Run on PikaPods](https://www.pikapods.com/static/run-button.svg)](https://www.pikapods.com/pods?run=rssbridge)
Patch/contributors :
The Heroku quick deploy currently does not work. It might work if you fork this repo and
modify the `repository` in `scalingo.json`. See https://github.com/RSS-Bridge/rss-bridge/issues/2688
* Yves ASTIER ([Draeli](https://github.com/Draeli)) : PHP optimizations, fixes, dynamic brigde/format list with all stuff behind and extend cache system. Mail : contact /at\ yves-astier.com
* [Mitsukarenai](https://github.com/Mitsukarenai) : Initial inspiration, collaborator
* [ArthurHoaro](https://github.com/ArthurHoaro)
* [BoboTiG](https://github.com/BoboTiG)
* [Astalaseven](https://github.com/Astalaseven)
* [qwertygc](https://github.com/qwertygc)
* [Djuuu](https://github.com/Djuuu)
* [Anadrark](https://github.com/Anadrark])
* [Grummfy](https://github.com/Grummfy)
* [Polopollo](https://github.com/Polopollo)
* [16mhz](https://github.com/16mhz)
* [kranack](https://github.com/kranack)
* [logmanoriginal](https://github.com/logmanoriginal)
* [polo2ro](https://github.com/polo2ro)
* [Riduidel](https://github.com/Riduidel)
* [superbaillot.net](http://superbaillot.net/)
* [vinzv](https://github.com/vinzv)
* [teromene](https://github.com/teromene)
* [nel50n](https://github.com/nel50n)
* [nyutag](https://github.com/nyutag)
* [ORelio](https://github.com/ORelio)
* [Pitchoule](https://github.com/Pitchoule)
* [pit-fgfjiudghdf](https://github.com/pit-fgfjiudghdf)
* [aledeg](https://github.com/aledeg)
* [alexAubin](https://github.com/alexAubin)
* [cnlpete](https://github.com/cnlpete)
* [corenting](https://github.com/corenting)
* [Daiyousei](https://github.com/Daiyousei)
* [erwang](https://github.com/erwang)
* [gsurrel](https://github.com/gsurrel)
* [kraoc](https://github.com/kraoc)
* [lagaisse](https://github.com/lagaisse)
* [az5he6ch](https://github.com/az5he6ch)
* [niawag](https://github.com/niawag)
* [JeremyRand](https://github.com/JeremyRand)
* [mro](https://github.com/mro)
Learn more in
[Installation](https://rss-bridge.github.io/rss-bridge/For_Hosts/Installation.html).
Licenses
===
Code is [Public Domain](UNLICENSE).
## How-to
Including `PHP Simple HTML DOM Parser` under the [MIT License](http://opensource.org/licenses/MIT)
### How to fix "Access denied."
Output is from php-fpm. It is unable to read index.php.
Technical notes
===
* There is a cache so that source services won't ban you even if you hammer the rss-bridge with requests. Each bridge can have a different duration for the cache. The `cache` subdirectory will be automatically created and cached objects older than 24 hours get purged.
* To implement a new Bridge, [follow the specifications](https://github.com/RSS-Bridge/rss-bridge/wiki/Bridge-API) and take a look at existing Bridges for examples.
* To enable debug mode (disabling cache and enabling error reporting), create an empty file named `DEBUG` in the root directory (next to `index.php`).
* For more information refer to the [Wiki](https://github.com/RSS-Bridge/rss-bridge/wiki)
chown rss-bridge:rss-bridge /var/www/rss-bridge/index.php
Rant
===
### How to password-protect the instance (token)
Modify `config.ini.php`:
[authentication]
token = "hunter2"
### How to remove all cache items
As current user:
bin/cache-clear
As user rss-bridge:
sudo -u rss-bridge bin/cache-clear
As root:
sudo bin/cache-clear
### How to remove all expired cache items
bin/cache-prune
### How to fix "PHP Fatal error: Uncaught Exception: The FileCache path is not writable"
```shell
# Give rss-bridge ownership
chown rss-bridge:rss-bridge -R /var/www/rss-bridge/cache
# Or, give www-data ownership
chown www-data:www-data -R /var/www/rss-bridge/cache
# Or, give everyone write permission
chmod 777 -R /var/www/rss-bridge/cache
# Or last ditch effort (CAREFUL)
rm -rf /var/www/rss-bridge/cache/ && mkdir /var/www/rss-bridge/cache/
```
### How to fix "attempt to write a readonly database"
The sqlite files (db, wal and shm) are not writeable.
chown -v rss-bridge:rss-bridge cache/*
### How to fix "Unable to prepare statement: 1, no such table: storage"
rm cache/*
### How to create a completely new bridge
New code files MUST have `declare(strict_types=1);` at the top of file:
```php
<?php
declare(strict_types=1);
```
Create the new bridge in e.g. `bridges/BearBlogBridge.php`:
```php
<?php
declare(strict_types=1);
class BearBlogBridge extends BridgeAbstract
{
const NAME = 'BearBlog (bearblog.dev)';
public function collectData()
{
$dom = getSimpleHTMLDOM('https://herman.bearblog.dev/blog/');
foreach ($dom->find('.blog-posts li') as $li) {
$a = $li->find('a', 0);
$this->items[] = [
'title' => $a->plaintext,
'uri' => 'https://herman.bearblog.dev' . $a->href,
];
}
}
}
```
Learn more in [bridge api](https://rss-bridge.github.io/rss-bridge/Bridge_API/index.html).
### How to enable all bridges
enabled_bridges[] = *
### How to enable some bridges
```
enabled_bridges[] = TwitchBridge
enabled_bridges[] = GettrBridge
```
### How to switch to memcached as cache backend
```
[cache]
; Cache backend: file (default), sqlite, memcached, null
type = "memcached"
```
### How to switch to sqlite3 as cache backend
type = "sqlite"
### How to disable bridge errors (as feed items)
When a bridge fails, RSS-Bridge will produce a feed with a single item describing the error.
This way, feed readers pick it up and you are notified.
If you don't want this behaviour, switch the error output to `http`:
[error]
; Defines how error messages are returned by RSS-Bridge
;
; "feed" = As part of the feed (default)
; "http" = As HTTP error message
; "none" = No errors are reported
output = "http"
### How to accumulate errors before finally reporting it
Modify `report_limit` so that an error must occur 3 times before it is reported.
; Defines how often an error must occur before it is reported to the user
report_limit = 3
The report count is reset to 0 each day.
### How to password-protect the instance (HTTP Basic Auth)
[authentication]
enable = true
username = "alice"
password = "cat"
Will typically require feed readers to be configured with the credentials.
It may also be possible to manually include the credentials in the URL:
https://alice:cat@rss-bridge.org/bridge01/?action=display&bridge=FabriceBellardBridge&format=Html
### How to create a new output format
See `formats/PlaintextFormat.php` for an example.
### How to run unit tests and linter
These commands require that you have installed the dev dependencies in `composer.json`.
Run all tests:
./vendor/bin/phpunit
Run a single test class:
./vendor/bin/phpunit --filter UrlTest
Run linter:
./vendor/bin/phpcs --standard=phpcs.xml --warning-severity=0 --extensions=php -p ./
https://github.com/squizlabs/PHP_CodeSniffer/wiki
### How to spawn a minimal development environment
php -S 127.0.0.1:9001
http://127.0.0.1:9001/
## Explanation
We are RSS-Bridge community, a group of developers continuing the project initiated by sebsauvage,
webmaster of
[sebsauvage.net](https://sebsauvage.net), author of
[Shaarli](https://sebsauvage.net/wiki/doku.php?id=php:shaarli) and
[ZeroBin](https://sebsauvage.net/wiki/doku.php?id=php:zerobin).
See [CONTRIBUTORS.md](CONTRIBUTORS.md)
RSS-Bridge uses caching to prevent services from banning your server for repeatedly updating feeds.
The specific cache duration can be different between bridges.
RSS-Bridge allows you to take full control over which bridges are displayed to the user.
That way you can host your own RSS-Bridge service with your favorite collection of bridges!
Current maintainers (as of 2024): @dvikan and @Mynacol #2519
## Reference
### Feed item structure
This is the feed item structure that bridges are expected to produce.
```php
$item = [
'uri' => 'https://example.com/blog/hello',
'title' => 'Hello world',
// Publication date in unix timestamp
'timestamp' => 1668706254,
'author' => 'Alice',
'content' => 'Here be item content',
'enclosures' => [
'https://example.com/foo.png',
'https://example.com/bar.png'
],
'categories' => [
'news',
'tech',
],
// Globally unique id
'uid' => 'e7147580c8747aad',
]
```
### Output formats
* `Atom`: Atom feed, for use in feed readers
* `Html`: Simple HTML page
* `Json`: JSON, for consumption by other applications
* `Mrss`: MRSS feed, for use in feed readers
* `Plaintext`: Raw text, for consumption by other applications
* `Sfeed`: Text, TAB separated
### Cache backends
* `File`
* `SQLite`
* `Memcached`
* `Array`
* `Null`
### Licenses
The source code for RSS-Bridge is [Public Domain](UNLICENSE).
RSS-Bridge uses third party libraries with their own license:
* [`Parsedown`](https://github.com/erusev/parsedown) licensed under the [MIT License](https://opensource.org/licenses/MIT)
* [`PHP Simple HTML DOM Parser`](https://simplehtmldom.sourceforge.io/docs/1.9/index.html) licensed under the [MIT License](https://opensource.org/licenses/MIT)
* [`php-urljoin`](https://github.com/fluffy-critter/php-urljoin) licensed under the [MIT License](https://opensource.org/licenses/MIT)
* [`Laravel framework`](https://github.com/laravel/framework/) licensed under the [MIT License](https://opensource.org/licenses/MIT)
## Rant
*Dear so-called "social" websites.*
Your catchword is "share", but you don't want us to share. You want to keep us within your walled gardens. That's why you've been removing RSS links from webpages, hiding them deep on your website, or removed feeds entirely, replacing it with crippled or demented proprietary API. **FUCK YOU.**
You're not social when you hamper sharing by removing feeds. You're happy to have customers creating content for your ecosystem, but you don't want this content out - a content you do not even own. Google Takeout is just a gimmick. We want our data to flow, we want RSS or ATOM feeds.
You're not social when you hamper sharing by removing feeds. You're happy to have customers creating content for your ecosystem, but you don't want this content out - a content you do not even own. Google Takeout is just a gimmick. We want our data to flow, we want RSS or Atom feeds.
We want to share with friends, using open protocols: RSS, ATOM, XMPP, whatever. Because no one wants to have *your* service with *your* applications using *your* API force-feeding them. Friends must be free to choose whatever software and service they want.
We want to share with friends, using open protocols: RSS, Atom, XMPP, whatever. Because no one wants to have *your* service with *your* applications using *your* API force-feeding them. Friends must be free to choose whatever software and service they want.
We are rebuilding bridges you have wilfully destroyed.
We are rebuilding bridges you have willfully destroyed.
Get your shit together: Put RSS/ATOM back in.
Get your shit together: Put RSS/Atom back in.

View File

@@ -0,0 +1,67 @@
<?php
/**
* Checks if the website for a given bridge is reachable.
*
* **Remarks**
* - This action is only available in debug mode.
* - Returns the bridge status as Json-formatted string.
* - Returns an error if the bridge is not whitelisted.
* - Returns a responsive web page that automatically checks all whitelisted
* bridges (using JavaScript) if no bridge is specified.
*/
class ConnectivityAction implements ActionInterface
{
private BridgeFactory $bridgeFactory;
public function __construct(
BridgeFactory $bridgeFactory
) {
$this->bridgeFactory = $bridgeFactory;
}
public function __invoke(Request $request): Response
{
if (Configuration::getConfig('system', 'env') !== 'dev') {
return new Response('This action is only available in dev environment!', 403);
}
$bridgeName = $request->get('bridge');
if (!$bridgeName) {
return new Response(render_template('connectivity.html.php'));
}
$bridgeClassName = $this->bridgeFactory->createBridgeClassName($bridgeName);
if (!$bridgeClassName) {
return new Response('Bridge not found', 404);
}
return $this->reportBridgeConnectivity($bridgeClassName);
}
private function reportBridgeConnectivity($bridgeClassName)
{
if (!$this->bridgeFactory->isEnabled($bridgeClassName)) {
throw new \Exception('Bridge is not whitelisted!');
}
$bridge = $this->bridgeFactory->create($bridgeClassName);
$curl_opts = [
CURLOPT_CONNECTTIMEOUT => 5,
CURLOPT_FOLLOWLOCATION => true,
];
$result = [
'bridge' => $bridgeClassName,
'successful' => false,
'http_code' => null,
];
try {
$response = getContents($bridge::URI, [], $curl_opts, true);
$result['http_code'] = $response->getCode();
if (in_array($result['http_code'], [200])) {
$result['successful'] = true;
}
} catch (\Exception $e) {
}
return new Response(Json::encode($result), 200, ['content-type' => 'text/json']);
}
}

51
actions/DetectAction.php Normal file
View File

@@ -0,0 +1,51 @@
<?php
class DetectAction implements ActionInterface
{
private BridgeFactory $bridgeFactory;
public function __construct(
BridgeFactory $bridgeFactory
) {
$this->bridgeFactory = $bridgeFactory;
}
public function __invoke(Request $request): Response
{
$url = $request->get('url');
$format = $request->get('format');
if (!$url) {
return new Response(render(__DIR__ . '/../templates/error.html.php', ['message' => 'You must specify a url']));
}
if (!$format) {
return new Response(render(__DIR__ . '/../templates/error.html.php', ['message' => 'You must specify a format']));
}
foreach ($this->bridgeFactory->getBridgeClassNames() as $bridgeClassName) {
if (!$this->bridgeFactory->isEnabled($bridgeClassName)) {
continue;
}
$bridge = $this->bridgeFactory->create($bridgeClassName);
$bridgeParams = $bridge->detectParameters($url);
if (!$bridgeParams) {
continue;
}
$query = [
'action' => 'display',
'bridge' => $bridgeClassName,
'format' => $format,
];
$query = array_merge($query, $bridgeParams);
return new Response('', 301, ['location' => '?' . http_build_query($query)]);
}
return new Response(render(__DIR__ . '/../templates/error.html.php', [
'message' => 'No bridge found for given URL: ' . $url,
]));
}
}

230
actions/DisplayAction.php Normal file
View File

@@ -0,0 +1,230 @@
<?php
class DisplayAction implements ActionInterface
{
private CacheInterface $cache;
private Logger $logger;
private BridgeFactory $bridgeFactory;
public function __construct(
CacheInterface $cache,
Logger $logger,
BridgeFactory $bridgeFactory
) {
$this->cache = $cache;
$this->logger = $logger;
$this->bridgeFactory = $bridgeFactory;
}
public function __invoke(Request $request): Response
{
$bridgeName = $request->get('bridge');
$format = $request->get('format');
$noproxy = $request->get('_noproxy');
if (!$bridgeName) {
return new Response(render(__DIR__ . '/../templates/error.html.php', ['message' => 'Missing bridge name parameter']), 400);
}
$bridgeClassName = $this->bridgeFactory->createBridgeClassName($bridgeName);
if (!$bridgeClassName) {
return new Response(render(__DIR__ . '/../templates/error.html.php', ['message' => 'Bridge not found']), 404);
}
if (!$format) {
return new Response(render(__DIR__ . '/../templates/error.html.php', ['message' => 'You must specify a format']), 400);
}
if (!$this->bridgeFactory->isEnabled($bridgeClassName)) {
return new Response(render(__DIR__ . '/../templates/error.html.php', ['message' => 'This bridge is not whitelisted']), 400);
}
// Disable proxy (if enabled and per user's request)
if (
Configuration::getConfig('proxy', 'url')
&& Configuration::getConfig('proxy', 'by_bridge')
&& $noproxy
) {
// This const is only used once in getContents()
define('NOPROXY', true);
}
$cacheKey = 'http_' . json_encode($request->toArray());
$bridge = $this->bridgeFactory->create($bridgeClassName);
$response = $this->createResponse($request, $bridge, $format);
if ($response->getCode() === 200) {
$ttl = $request->get('_cache_timeout');
if (Configuration::getConfig('cache', 'custom_timeout') && $ttl) {
$ttl = (int) $ttl;
} else {
$ttl = $bridge->getCacheTimeout();
}
$this->cache->set($cacheKey, $response, $ttl);
}
return $response;
}
private function createResponse(Request $request, BridgeAbstract $bridge, string $format)
{
$items = [];
try {
$bridge->loadConfiguration();
// Remove parameters that don't concern bridges
$remove = [
'token',
'action',
'bridge',
'format',
'_noproxy',
'_cache_timeout',
'_error_time',
'_', // Some RSS readers add a cache-busting parameter (_=<timestamp>) to feed URLs, detect and ignore them.
];
$requestArray = $request->toArray();
$input = array_diff_key($requestArray, array_fill_keys($remove, ''));
$bridge->setInput($input);
$bridge->collectData();
$items = $bridge->getItems();
} catch (\Throwable $e) {
if ($e instanceof ClientException) {
$this->logger->debug(sprintf('Exception in DisplayAction(%s): %s', $bridge->getShortName(), create_sane_exception_message($e)));
} elseif ($e instanceof RateLimitException) {
$this->logger->debug(sprintf('Exception in DisplayAction(%s): %s', $bridge->getShortName(), create_sane_exception_message($e)));
return new Response(render(__DIR__ . '/../templates/exception.html.php', ['e' => $e]), 429);
} elseif ($e instanceof HttpException) {
if (in_array($e->getCode(), [429, 503])) {
// Log with debug, immediately reproduce and return
$this->logger->debug(sprintf('Exception in DisplayAction(%s): %s', $bridge->getShortName(), create_sane_exception_message($e)));
return new Response(render(__DIR__ . '/../templates/exception.html.php', ['e' => $e]), $e->getCode());
}
// Some other status code which we let fail normally (but don't log it)
} else {
$this->logger->error(sprintf('Exception in DisplayAction(%s)', $bridge->getShortName()), ['e' => $e]);
}
$errorOutput = Configuration::getConfig('error', 'output');
$reportLimit = Configuration::getConfig('error', 'report_limit');
$errorCount = 1;
if ($reportLimit > 1) {
$errorCount = $this->logBridgeError($bridge->getName(), $e->getCode());
}
// Let clients know about the error if we are passed the report limit
if ($errorCount >= $reportLimit) {
if ($errorOutput === 'feed') {
// Render the exception as a feed item
$items = [$this->createFeedItemFromException($e, $bridge)];
} elseif ($errorOutput === 'http') {
return new Response(render(__DIR__ . '/../templates/exception.html.php', ['e' => $e]), 500);
} elseif ($errorOutput === 'none') {
// Do nothing (produces an empty feed)
}
}
}
$formatFactory = new FormatFactory();
$format = $formatFactory->create($format);
$format->setItems($items);
$format->setFeed($bridge->getFeed());
$now = time();
$format->setLastModified($now);
$headers = [
'last-modified' => gmdate('D, d M Y H:i:s ', $now) . 'GMT',
'content-type' => $format->getMimeType() . '; charset=UTF-8',
];
$body = $format->render();
// This is supposed to remove non-utf8 byte sequences, but I'm unsure if it works
ini_set('mbstring.substitute_character', 'none');
$body = mb_convert_encoding($body, 'UTF-8', 'UTF-8');
return new Response($body, 200, $headers);
}
private function createFeedItemFromException($e, BridgeAbstract $bridge): array
{
$item = [];
// Create a unique identifier every 24 hours
$uniqueIdentifier = urlencode((int)(time() / 86400));
$title = sprintf('Bridge returned error %s! (%s)', $e->getCode(), $uniqueIdentifier);
$item['title'] = $title;
$item['uri'] = get_current_url();
$item['timestamp'] = time();
// Create an item identifier for feed readers e.g. "staysafetv twitch videos_19389"
$item['uid'] = $bridge->getName() . '_' . $uniqueIdentifier;
$content = render_template(__DIR__ . '/../templates/bridge-error.html.php', [
'error' => render_template(__DIR__ . '/../templates/exception.html.php', ['e' => $e]),
'searchUrl' => self::createGithubSearchUrl($bridge),
'issueUrl' => self::createGithubIssueUrl($bridge, $e),
'maintainer' => $bridge->getMaintainer(),
]);
$item['content'] = $content;
return $item;
}
private function logBridgeError($bridgeName, $code)
{
// todo: it's not really necessary to json encode $report
$cacheKey = 'error_reporting_' . $bridgeName . '_' . $code;
$report = $this->cache->get($cacheKey);
if ($report) {
$report = Json::decode($report);
$report['time'] = time();
$report['count']++;
} else {
$report = [
'error' => $code,
'time' => time(),
'count' => 1,
];
}
$ttl = 86400 * 5;
$this->cache->set($cacheKey, Json::encode($report), $ttl);
return $report['count'];
}
private static function createGithubIssueUrl(BridgeAbstract $bridge, \Throwable $e): string
{
$maintainer = $bridge->getMaintainer();
if (str_contains($maintainer, ',')) {
$maintainers = explode(',', $maintainer);
} else {
$maintainers = [$maintainer];
}
$maintainers = array_map('trim', $maintainers);
$queryString = $_SERVER['QUERY_STRING'] ?? '';
$query = [
'title' => $bridge->getName() . ' failed with: ' . $e->getMessage(),
'body' => sprintf(
"```\n%s\n\n%s\n\nQuery string: %s\nVersion: %s\nOs: %s\nPHP version: %s\n```\nMaintainer: @%s",
create_sane_exception_message($e),
implode("\n", trace_to_call_points(trace_from_exception($e))),
$queryString,
Configuration::getVersion(),
PHP_OS_FAMILY,
phpversion() ?: 'Unknown',
implode(', @', $maintainers),
),
'labels' => 'Bridge-Broken',
'assignee' => $maintainer[0],
];
return 'https://github.com/RSS-Bridge/rss-bridge/issues/new?' . http_build_query($query);
}
private static function createGithubSearchUrl($bridge): string
{
return sprintf(
'https://github.com/RSS-Bridge/rss-bridge/issues?q=%s',
urlencode('is:issue is:open ' . $bridge->getName())
);
}
}

View File

@@ -0,0 +1,95 @@
<?php
/**
* This action is used by the frontpage form search.
* It finds a bridge based off of a user input url.
* It uses bridges' detectParameters implementation.
*/
class FindfeedAction implements ActionInterface
{
private BridgeFactory $bridgeFactory;
public function __construct(
BridgeFactory $bridgeFactory
) {
$this->bridgeFactory = $bridgeFactory;
}
public function __invoke(Request $request): Response
{
$url = $request->get('url');
$format = $request->get('format');
if (!$url) {
return new Response('You must specify a url', 400);
}
if (!$format) {
return new Response('You must specify a format', 400);
}
$results = [];
foreach ($this->bridgeFactory->getBridgeClassNames() as $bridgeClassName) {
if (!$this->bridgeFactory->isEnabled($bridgeClassName)) {
continue;
}
$bridge = $this->bridgeFactory->create($bridgeClassName);
$bridgeParams = $bridge->detectParameters($url);
if ($bridgeParams === null) {
continue;
}
// It's allowed to have no 'context' in a bridge (only a default context without any name)
// In this case, the reference to the parameters are found in the first element of the PARAMETERS array
$context = $bridgeParams['context'] ?? 0;
$bridgeData = [];
// Construct the array of parameters
foreach ($bridgeParams as $key => $value) {
// 'context' is a special case : it's a bridge parameters, there is no "name" for this parameter
if ($key == 'context') {
$bridgeData[$key]['name'] = 'Context';
$bridgeData[$key]['value'] = $value;
} else {
$bridgeData[$key]['name'] = $this->getParameterName($bridge, $context, $key);
$bridgeData[$key]['value'] = $value;
}
}
$bridgeParams['bridge'] = $bridgeClassName;
$bridgeParams['format'] = $format;
$content = [
'url' => './?action=display&' . http_build_query($bridgeParams),
'bridgeParams' => $bridgeParams,
'bridgeData' => $bridgeData,
'bridgeMeta' => [
'name' => $bridge::NAME,
'description' => $bridge::DESCRIPTION,
'parameters' => $bridge::PARAMETERS,
'icon' => $bridge->getIcon(),
],
];
$results[] = $content;
}
if ($results === []) {
return new Response(Json::encode(['message' => 'No bridge found for given url']), 404, ['content-type' => 'application/json']);
}
return new Response(Json::encode($results), 200, ['content-type' => 'application/json']);
}
// Get parameter name in the actual context, or in the global parameter
private function getParameterName($bridge, $context, $key)
{
if (isset($bridge::PARAMETERS[$context][$key]['name'])) {
$name = $bridge::PARAMETERS[$context][$key]['name'];
} else if (isset($bridge::PARAMETERS['global'][$key]['name'])) {
$name = $bridge::PARAMETERS['global'][$key]['name'];
} else {
$name = 'Variable "' . $key . '" (No name provided)';
}
return $name;
}
}

View File

@@ -0,0 +1,49 @@
<?php
final class FrontpageAction implements ActionInterface
{
private BridgeFactory $bridgeFactory;
public function __construct(
BridgeFactory $bridgeFactory
) {
$this->bridgeFactory = $bridgeFactory;
}
public function __invoke(Request $request): Response
{
$token = $request->getAttribute('token');
$messages = [];
$activeBridges = 0;
$bridgeClassNames = $this->bridgeFactory->getBridgeClassNames();
foreach ($this->bridgeFactory->getMissingEnabledBridges() as $missingEnabledBridge) {
$messages[] = [
'body' => sprintf('Warning : Bridge "%s" not found', $missingEnabledBridge),
'level' => 'warning'
];
}
$body = '';
foreach ($bridgeClassNames as $bridgeClassName) {
if ($this->bridgeFactory->isEnabled($bridgeClassName)) {
$body .= BridgeCard::render($this->bridgeFactory, $bridgeClassName, $token);
$activeBridges++;
}
}
$response = new Response(render(__DIR__ . '/../templates/frontpage.html.php', [
'messages' => $messages,
'admin_email' => Configuration::getConfig('admin', 'email'),
'admin_telegram' => Configuration::getConfig('admin', 'telegram'),
'bridges' => $body,
'active_bridges' => $activeBridges,
'total_bridges' => count($bridgeClassNames),
]));
// TODO: The rendered template could be cached, but beware config changes that changes the html
return $response;
}
}

15
actions/HealthAction.php Normal file
View File

@@ -0,0 +1,15 @@
<?php
declare(strict_types=1);
class HealthAction implements ActionInterface
{
public function __invoke(Request $request): Response
{
$response = [
'code' => 200,
'message' => 'all is good',
];
return new Response(Json::encode($response), 200, ['content-type' => 'application/json']);
}
}

36
actions/ListAction.php Normal file
View File

@@ -0,0 +1,36 @@
<?php
class ListAction implements ActionInterface
{
private BridgeFactory $bridgeFactory;
public function __construct(
BridgeFactory $bridgeFactory
) {
$this->bridgeFactory = $bridgeFactory;
}
public function __invoke(Request $request): Response
{
$list = new \stdClass();
$list->bridges = [];
$list->total = 0;
foreach ($this->bridgeFactory->getBridgeClassNames() as $bridgeClassName) {
$bridge = $this->bridgeFactory->create($bridgeClassName);
$list->bridges[$bridgeClassName] = [
'status' => $this->bridgeFactory->isEnabled($bridgeClassName) ? 'active' : 'inactive',
'uri' => $bridge->getURI(),
'donationUri' => $bridge->getDonationURI(),
'name' => $bridge->getName(),
'icon' => $bridge->getIcon(),
'parameters' => $bridge->getParameters(),
'maintainer' => $bridge->getMaintainer(),
'description' => $bridge->getDescription()
];
}
$list->total = count($list->bridges);
return new Response(Json::encode($list), 200, ['content-type' => 'application/json']);
}
}

8
app.json Normal file
View File

@@ -0,0 +1,8 @@
{
"service": "Heroku",
"name": "rss-bridge-heroku",
"description": "RSS-Bridge is a PHP project capable of generating RSS and Atom feeds for websites which don't have one.",
"repository": "https://github.com/RSS-Bridge/rss-bridge?1651005770",
"keywords": ["php", "rss-bridge", "rss"]
}

16
bin/cache-clear Executable file
View File

@@ -0,0 +1,16 @@
#!/usr/bin/env php
<?php
/**
* Remove all items from the cache
*/
require __DIR__ . '/../lib/bootstrap.php';
require __DIR__ . '/../lib/config.php';
$container = require __DIR__ . '/../lib/dependencies.php';
/** @var CacheInterface $cache */
$cache = $container['cache'];
$cache->clear();

24
bin/cache-prune Executable file
View File

@@ -0,0 +1,24 @@
#!/usr/bin/env php
<?php
/**
* Remove all expired items from the cache
*/
require __DIR__ . '/../lib/bootstrap.php';
require __DIR__ . '/../lib/config.php';
$container = require __DIR__ . '/../lib/dependencies.php';
if (
Configuration::getConfig('cache', 'type') === 'file'
&& !Configuration::getConfig('FileCache', 'enable_purge')
) {
// Override enable_purge for this particular execution
Configuration::setConfig('FileCache', 'enable_purge', true);
}
/** @var CacheInterface $cache */
$cache = $container['cache'];
$cache->prune();

20
bin/test Executable file
View File

@@ -0,0 +1,20 @@
#!/usr/bin/env php
<?php
/**
* Add log records to all three levels (for testing purposes)
*/
require __DIR__ . '/../lib/bootstrap.php';
require __DIR__ . '/../lib/config.php';
$container = require __DIR__ . '/../lib/dependencies.php';
/** @var Logger $logger */
$logger = $container['logger'];
$logger->debug('This is a test debug message');
$logger->info('This is a test info message');
$logger->error('This is a test error message');

49
bridges/ABCNewsBridge.php Normal file
View File

@@ -0,0 +1,49 @@
<?php
class ABCNewsBridge extends BridgeAbstract
{
const NAME = 'ABC News Bridge';
const URI = 'https://www.abc.net.au';
const DESCRIPTION = 'Topics of the Australian Broadcasting Corporation';
const MAINTAINER = 'yue-dongchen';
const PARAMETERS = [
[
'topic' => [
'type' => 'list',
'name' => 'Region',
'title' => 'Choose state',
'values' => [
'ACT' => 'act',
'NSW' => 'nsw',
'NT' => 'nt',
'QLD' => 'qld',
'SA' => 'sa',
'TAS' => 'tas',
'VIC' => 'vic',
'WA' => 'wa'
],
]
]
];
public function collectData()
{
$url = sprintf('https://www.abc.net.au/news/%s', $this->getInput('topic'));
$dom = getSimpleHTMLDOM($url);
$dom = $dom->find('div[data-component="PaginationList"]', 0);
if (!$dom) {
throw new \Exception(sprintf('Unable to find css selector on `%s`', $url));
}
$dom = defaultLinkTo($dom, $this->getURI());
foreach ($dom->find('article[data-component="DetailCard"]') as $article) {
$a = $article->find('a', 0);
$this->items[] = [
'title' => $a->plaintext,
'uri' => $a->href,
'content' => $article->find('p', 0)->plaintext,
'timestamp' => strtotime($article->find('time', 0)->datetime),
];
}
}
}

View File

@@ -1,42 +0,0 @@
<?php
class ABCTabsBridge extends BridgeAbstract {
const MAINTAINER = 'kranack';
const NAME = 'ABC Tabs Bridge';
const URI = 'https://www.abc-tabs.com/';
const DESCRIPTION = 'Returns 22 newest tabs';
public function collectData(){
$html = '';
$html = getSimpleHTMLDOM(static::URI.'tablatures/nouveautes.html')
or returnClientError('No results for this query.');
$table = $html->find('table#myTable', 0)->children(1);
foreach ($table->find('tr') as $tab) {
$item = array();
$item['author'] = $tab->find('td', 1)->plaintext
. ' - '
. $tab->find('td', 2)->plaintext;
$item['title'] = $tab->find('td', 1)->plaintext
. ' - '
. $tab->find('td', 2)->plaintext;
$item['content'] = 'Le '
. $tab->find('td', 0)->plaintext
. '<br> Par: '
. $tab->find('td', 5)->plaintext
. '<br> Type: '
. $tab->find('td', 3)->plaintext;
$item['id'] = static::URI
. $tab->find('td', 2)->find('a', 0)->getAttribute('href');
$item['uri'] = static::URI
. $tab->find('td', 2)->find('a', 0)->getAttribute('href');
$this->items[] = $item;
}
}
}

116
bridges/ABolaBridge.php Normal file
View File

@@ -0,0 +1,116 @@
<?php
class ABolaBridge extends BridgeAbstract
{
const NAME = 'A Bola';
const URI = 'https://abola.pt/';
const DESCRIPTION = 'Returns news from the Portuguese sports newspaper A BOLA.PT';
const MAINTAINER = 'rmscoelho';
const CACHE_TIMEOUT = 3600;
const PARAMETERS = [
[
'feed' => [
'name' => 'News Feed',
'type' => 'list',
'title' => 'Feeds from the Portuguese sports newspaper A BOLA.PT',
'values' => [
'Últimas' => 'Nnh/Noticias',
'Seleção Nacional' => 'Selecao/Noticias',
'Futebol Nacional' => [
'Notícias' => 'Nacional/Noticias',
'Primeira Liga' => 'Nacional/Liga/Noticias',
'Liga 2' => 'Nacional/Liga2/Noticias',
'Liga 3' => 'Nacional/Liga3/Noticias',
'Liga Revelação' => 'Nacional/Liga-Revelacao/Noticias',
'Campeonato de Portugal' => 'Nacional/Campeonato-Portugal/Noticias',
'Distritais' => 'Nacional/Distritais/Noticias',
'Taça de Portugal' => 'Nacional/TPortugal/Noticias',
'Futebol Feminino' => 'Nacional/FFeminino/Noticias',
'Futsal' => 'Nacional/Futsal/Noticias',
],
'Futebol Internacional' => [
'Notícias' => 'Internacional/Noticias/Noticias',
'Liga dos Campeões' => 'Internacional/Liga-dos-campeoes/Noticias',
'Liga Europa' => 'Internacional/Liga-europa/Noticias',
'Liga Conferência' => 'Internacional/Liga-conferencia/Noticias',
'Liga das Nações' => 'Internacional/Liga-das-nacoes/Noticias',
'UEFA Youth League' => 'Internacional/Uefa-Youth-League/Noticias',
],
'Mercado' => 'Mercado',
'Modalidades' => 'Modalidades/Noticias',
'Motores' => 'Motores/Noticias',
]
]
]
];
public function getIcon()
{
return 'https://abola.pt/img/icons/favicon-96x96.png';
}
public function getName()
{
return !is_null($this->getKey('feed')) ? self::NAME . ' | ' . $this->getKey('feed') : self::NAME;
}
public function getURI()
{
return self::URI . $this->getInput('feed');
}
public function collectData()
{
$url = sprintf('https://abola.pt/%s', $this->getInput('feed'));
$dom = getSimpleHTMLDOM($url);
if ($this->getInput('feed') !== 'Mercado') {
$dom = $dom->find('div#body_Todas1_upNoticiasTodas', 0);
} else {
$dom = $dom->find('div#body_NoticiasMercado_upNoticiasTodas', 0);
}
if (!$dom) {
throw new \Exception(sprintf('Unable to find css selector on `%s`', $url));
}
$dom = defaultLinkTo($dom, $this->getURI());
foreach ($dom->find('div.media') as $key => $article) {
//Get thumbnail
$image = $article->find('.media-img', 0)->style;
$image = preg_replace('/background-image: url\(/i', '', $image);
$image = substr_replace($image, '', -4);
$image = preg_replace('/https:\/\//i', '', $image);
$image = preg_replace('/www\./i', '', $image);
$image = preg_replace('/\/\//', '/', $image);
$image = preg_replace('/\/\/\//', '//', $image);
$image = substr($image, 7);
$image = 'https://' . $image;
$image = preg_replace('/ptimg/', 'pt/img', $image);
$image = preg_replace('/\/\/bola/', 'www.abola', $image);
//Timestamp
$date = date('Y/m/d');
if (!is_null($article->find("span#body_Todas1_rptNoticiasTodas_lblData_$key", 0))) {
$date = $article->find("span#body_Todas1_rptNoticiasTodas_lblData_$key", 0)->plaintext;
$date = preg_replace('/\./', '/', $date);
}
$time = $article->find("span#body_Todas1_rptNoticiasTodas_lblHora_$key", 0)->plaintext;
$date = explode('/', $date);
$time = explode(':', $time);
$year = $date[0];
$month = $date[1];
$day = $date[2];
$hour = $time[0];
$minute = $time[1];
$timestamp = mktime($hour, $minute, 0, $month, $day, $year);
//Content
$image = '<img src="' . $image . '" alt="' . $article->find('h4 span', 0)->plaintext . '" />';
$description = '<p>' . $article->find('.media-texto > span', 0)->plaintext . '</p>';
$content = $image . '</br>' . $description;
$a = $article->find('.media-body > a', 0);
$this->items[] = [
'title' => $a->find('h4 span', 0)->plaintext,
'uri' => $a->href,
'content' => $content,
'timestamp' => $timestamp,
];
}
}
}

250
bridges/AO3Bridge.php Normal file
View File

@@ -0,0 +1,250 @@
<?php
class AO3Bridge extends BridgeAbstract
{
const NAME = 'AO3';
const URI = 'https://archiveofourown.org/';
const CACHE_TIMEOUT = 1800;
const DESCRIPTION = 'Returns works or chapters from Archive of Our Own';
const MAINTAINER = 'Obsidienne';
const PARAMETERS = [
'List' => [
'url' => [
'name' => 'url',
'required' => true,
// Example: F/F tag
'exampleValue' => 'https://archiveofourown.org/tags/F*s*F/works',
],
'range' => [
'name' => 'Chapter Content',
'title' => 'Chapter(s) to include in each work\'s feed entry',
'defaultValue' => null,
'type' => 'list',
'values' => [
'None' => null,
'First' => 'first',
'Latest' => 'last',
'Entire work' => 'all',
],
],
'unique' => [
'name' => 'Make separate entries for new fic chapters',
'type' => 'checkbox',
'required' => false,
'title' => 'Make separate entries for new fic chapters',
'defaultValue' => 'checked',
],
'limit' => self::LIMIT,
],
'Bookmarks' => [
'user' => [
'name' => 'user',
'required' => true,
// Example: Nyaaru's bookmarks
'exampleValue' => 'Nyaaru',
],
],
'Work' => [
'id' => [
'name' => 'id',
'required' => true,
// Example: latest chapters from A Better Past by LysSerris
'exampleValue' => '18181853',
],
]
];
private $title;
public function collectData()
{
switch ($this->queriedContext) {
case 'Bookmarks':
$this->collectList($this->getURI());
break;
case 'List':
$this->collectList($this->getURI());
break;
case 'Work':
$this->collectWork($this->getURI());
break;
}
}
/**
* Feed for lists of works (e.g. recent works, search results, filtered tags,
* bookmarks, series, collections).
*/
private function collectList($url)
{
$version = 'v0.0.1';
$headers = [
"useragent: rss-bridge $version (https://github.com/RSS-Bridge/rss-bridge)"
];
$response = getContents($url, $headers);
$html = \str_get_html($response);
$html = defaultLinkTo($html, self::URI);
// Get list title. Will include page range + count in some cases
$heading = ($html->find('#main h2', 0));
if ($heading->find('a.tag')) {
$heading = $heading->find('a.tag', 0);
}
$this->title = $heading->plaintext;
$limit = $this->getInput('limit') ?? 3;
$count = 0;
foreach ($html->find('.index.group > li') as $element) {
$item = [];
$title = $element->find('div h4 a', 0);
if (!isset($title)) {
continue; // discard deleted works
}
$item['title'] = $title->plaintext;
$item['uri'] = $title->href;
$strdate = $element->find('div p.datetime', 0)->plaintext;
$item['timestamp'] = strtotime($strdate);
// detach from rest of page because remove() is buggy
$element = str_get_html($element->outertext());
$tags = $element->find('ul.required-tags', 0);
foreach ($tags->childNodes() as $tag) {
$item['categories'][] = html_entity_decode($tag->plaintext);
}
$tags->remove();
$tags = $element->find('ul.tags', 0);
foreach ($tags->childNodes() as $tag) {
$item['categories'][] = html_entity_decode($tag->plaintext);
}
$tags->remove();
$item['content'] = implode('', $element->childNodes());
$chapters = $element->find('dl dd.chapters', 0);
// bookmarked series and external works do not have a chapters count
$chapters = (isset($chapters) ? $chapters->plaintext : 0);
if ($this->getInput('unique')) {
$item['uid'] = $item['uri'] . "/$strdate/$chapters";
} else {
$item['uid'] = $item['uri'];
}
// Fetch workskin of desired chapter(s) in list
if ($this->getInput('range') && ($limit == 0 || $count++ < $limit)) {
$url = $item['uri'];
switch ($this->getInput('range')) {
case ('all'):
$url .= '?view_full_work=true';
break;
case ('first'):
break;
case ('last'):
// only way to get this is using the navigate page unfortunately
$url .= '/navigate';
$response = getContents($url, $headers);
$html = \str_get_html($response);
$html = defaultLinkTo($html, self::URI);
$url = $html->find('ol.index.group > li > a', -1)->href;
break;
}
$response = getContents($url, $headers);
$html = \str_get_html($response);
$html = defaultLinkTo($html, self::URI);
// remove duplicate fic summary
if ($ficsum = $html->find('#workskin > .preface > .summary', 0)) {
$ficsum->remove();
}
$item['content'] .= $html->find('#workskin', 0);
}
// Use predictability of download links to generate enclosures
$wid = explode('/', $item['uri'])[4];
foreach (['azw3', 'epub', 'mobi', 'pdf', 'html'] as $ext) {
$item['enclosures'][] = 'https://archiveofourown.org/downloads/' . $wid . '/work.' . $ext;
}
$this->items[] = $item;
}
}
/**
* Feed for recent chapters of a specific work.
*/
private function collectWork($url)
{
$version = 'v0.0.1';
$headers = [
"useragent: rss-bridge $version (https://github.com/RSS-Bridge/rss-bridge)"
];
$response = getContents($url . '/navigate', $headers);
$html = \str_get_html($response);
$html = defaultLinkTo($html, self::URI);
$response = getContents($url . '?view_full_work=true', $headers);
$workhtml = \str_get_html($response);
$workhtml = defaultLinkTo($workhtml, self::URI);
$this->title = $html->find('h2 a', 0)->plaintext;
$nav = $html->find('ol.index.group > li');
for ($i = 0; $i < count($nav); $i++) {
$item = [];
$element = $nav[$i];
$item['title'] = $element->find('a', 0)->plaintext;
$item['content'] = $workhtml->find('#chapter-' . ($i + 1), 0);
$item['uri'] = $element->find('a', 0)->href;
$strdate = $element->find('span.datetime', 0)->plaintext;
$strdate = str_replace('(', '', $strdate);
$strdate = str_replace(')', '', $strdate);
$item['timestamp'] = strtotime($strdate);
$item['uid'] = $item['uri'] . "/$strdate";
$this->items[] = $item;
}
$this->items = array_reverse($this->items);
}
public function getName()
{
$name = parent::getName() . " $this->queriedContext";
if (isset($this->title)) {
$name .= " - $this->title";
}
return $name;
}
public function getIcon()
{
return self::URI . '/favicon.ico';
}
public function getURI()
{
$url = parent::getURI();
switch ($this->queriedContext) {
case 'Bookmarks':
$user = $this->getInput('user');
$url = self::URI
. '/users/' . $user
. '/bookmarks?bookmark_search[sort_column]=bookmarkable_date';
break;
case 'List':
$url = $this->getInput('url');
break;
case 'Work':
$url = self::URI . '/works/' . $this->getInput('id');
break;
}
return $url;
}
}

View File

@@ -0,0 +1,173 @@
<?php
class ARDAudiothekBridge extends BridgeAbstract
{
const NAME = 'ARD-Audiothek Bridge';
const URI = 'https://www.ardaudiothek.de';
const DESCRIPTION = 'Feed of any show in the ARD-Audiothek, specified by its path';
const MAINTAINER = 'Mar-Koeh';
/*
* The URL Prefix of the API
* @const APIENDPOINT https-URL of the used endpoint, ending in `/`
*/
const APIENDPOINT = 'https://api.ardaudiothek.de/';
/*
* The requested width of the preview image
* 448 and 128 have been observed on the wild
* @const IMAGEWIDTH width in px of the preview image
*/
const IMAGEWIDTH = 448;
/*
* Placeholder that will be replace by IMAGEWIDTH in the preview image URL
* @const IMAGEWIDTHPLACEHOLDER
*/
const IMAGEWIDTHPLACEHOLDER = '{width}';
/*
* File extension appended to image link in $this->icon
* @const IMAGEEXTENSION
*/
const IMAGEEXTENSION = '.jpg';
const PARAMETERS = [
[
'path' => [
'name' => 'Show Link or ID',
'required' => true,
'title' => 'Link to the show page or just its numeric suffix',
'defaultValue' => 'https://www.ardaudiothek.de/sendung/kalk-welk/10777871/'
],
'limit' => self::LIMIT,
]
];
/**
* Holds the title of the current show
*
* @var string
*/
private $title;
/**
* Holds the URI of the show
*
* @var string
*/
private $uri;
/**
* Holds the icon of the feed
*
*/
private $icon;
public function collectData()
{
$path = $this->getInput('path');
$limit = $this->getInput('limit');
$oldTz = date_default_timezone_get();
date_default_timezone_set('Europe/Berlin');
$pathComponents = explode('/', $path);
if (empty($pathComponents)) {
throwClientException('Path may not be empty');
}
if (count($pathComponents) < 2) {
$showID = $pathComponents[0];
} else {
$lastKey = count($pathComponents) - 1;
$showID = $pathComponents[$lastKey];
if (strlen($showID) === 0) {
$showID = $pathComponents[$lastKey - 1];
}
}
$url = self::APIENDPOINT . 'programsets/' . $showID . '/';
$json1 = getContents($url);
$data1 = Json::decode($json1, false);
$processedJSON = $data1->data->programSet;
if (!$processedJSON) {
throw new \Exception('Unable to find show id: ' . $showID);
}
$answerLength = 1;
$offset = 0;
$numberOfElements = 1;
while ($answerLength != 0 && $offset < $numberOfElements && (is_null($limit) || $offset < $limit)) {
$json2 = getContents($url . '?offset=' . $offset);
$data2 = Json::decode($json2, false);
$processedJSON = $data2->data->programSet;
$answerLength = count($processedJSON->items->nodes);
$offset = $offset + $answerLength;
$numberOfElements = $processedJSON->numberOfElements;
foreach ($processedJSON->items->nodes as $audio) {
$item = [];
$item['uri'] = $audio->sharingUrl;
$item['title'] = $audio->title;
$imageSquare = str_replace(self::IMAGEWIDTHPLACEHOLDER, self::IMAGEWIDTH, $audio->image->url1X1);
$image = str_replace(self::IMAGEWIDTHPLACEHOLDER, self::IMAGEWIDTH, $audio->image->url);
$item['enclosures'] = [
$audio->audios[0]->url,
$imageSquare
];
// synopsis in list is shortened, full synopsis is available using one request per item
$item['content'] = '<img src="' . $image . '" /><p>' . $audio->synopsis . '</p>';
$item['timestamp'] = $audio->publicationStartDateAndTime;
$item['uid'] = $audio->id;
$item['author'] = $audio->programSet->publicationService->title;
$category = $audio->programSet->editorialCategories->title ?? null;
if ($category) {
$item['categories'] = [$category];
}
$item['itunes'] = [
'duration' => $audio->duration,
];
$this->items[] = $item;
}
}
$this->title = $processedJSON->title;
$this->uri = $processedJSON->sharingUrl;
$this->icon = str_replace(self::IMAGEWIDTHPLACEHOLDER, self::IMAGEWIDTH, $processedJSON->image->url1X1);
// add image file extension to URL so icon is shown in generated RSS feeds, see
// https://github.com/RSS-Bridge/rss-bridge/blob/4aed05c7b678b5673386d61374bba13637d15487/formats/MrssFormat.php#L76
$this->icon = $this->icon . self::IMAGEEXTENSION;
$this->items = array_slice($this->items, 0, $limit);
date_default_timezone_set($oldTz);
}
/** {@inheritdoc} */
public function getURI()
{
if (!empty($this->uri)) {
return $this->uri;
}
return parent::getURI();
}
/** {@inheritdoc} */
public function getName()
{
if (!empty($this->title)) {
return $this->title;
}
return parent::getName();
}
/** {@inheritdoc} */
public function getIcon()
{
if (!empty($this->icon)) {
return $this->icon;
}
return parent::getIcon();
}
}

View File

@@ -0,0 +1,114 @@
<?php
class ARDMediathekBridge extends BridgeAbstract
{
const NAME = 'ARD-Mediathek Bridge';
const URI = 'https://www.ardmediathek.de';
const DESCRIPTION = 'Feed of any series in the ARD-Mediathek, specified by its path';
const MAINTAINER = 'yue-dongchen';
/*
* Number of Items to be requested from ARDmediathek API
* 12 has been observed on the wild
* 29 is the highest successfully tested value
* More Items could be fetched via pagination
* The JSON-field pagination holds more information on that
* @const PAGESIZE number of requested items
*/
const PAGESIZE = 29;
/*
* The URL Prefix of the (Webapp-)API
* @const APIENDPOINT https-URL of the used endpoint
*/
const APIENDPOINT = 'https://api.ardmediathek.de/page-gateway/widgets/ard/asset/';
/*
* The URL prefix of the video link
* URLs from the webapp include a slug containing titles of show, episode, and tv station.
* It seems to work without that.
* @const VIDEOLINKPREFIX https-URL prefix of video links
*/
const VIDEOLINKPREFIX = 'https://www.ardmediathek.de/video/';
/*
* The requested width of the preview image
* 432 has been observed on the wild
* The webapp seems to also compute and add the height value
* It seems to works without that.
* @const IMAGEWIDTH width in px of the preview image
*/
const IMAGEWIDTH = 432;
/*
* Placeholder that will be replace by IMAGEWIDTH in the preview image URL
* @const IMAGEWIDTHPLACEHOLDER
*/
const IMAGEWIDTHPLACEHOLDER = '{width}';
/**
* Title of the current show
* @var string
*/
private $title;
const PARAMETERS = [
[
'path' => [
'name' => 'Show Link or ID',
'required' => true,
'title' => 'Link to the show page or just its alphanumeric suffix',
'defaultValue' => 'https://www.ardmediathek.de/sendung/45-min/Y3JpZDovL25kci5kZS8xMzkx/'
]
]
];
public function collectData()
{
$oldTz = date_default_timezone_get();
date_default_timezone_set('Europe/Berlin');
$pathComponents = explode('/', $this->getInput('path'));
if (empty($pathComponents)) {
throwClientException('Path may not be empty');
}
if (count($pathComponents) < 2) {
$showID = $pathComponents[0];
} else {
$lastKey = count($pathComponents) - 1;
$showID = $pathComponents[$lastKey];
if (strlen($showID) === 0) {
$showID = $pathComponents[$lastKey - 1];
}
}
$url = self::APIENDPOINT . $showID . '?pageSize=' . self::PAGESIZE;
$rawJSON = getContents($url);
$processedJSON = json_decode($rawJSON);
foreach ($processedJSON->teasers as $video) {
$item = [];
// there is also ->links->self->id, ->links->self->urlId, ->links->target->id, ->links->target->urlId
$item['uri'] = self::VIDEOLINKPREFIX . $video->id . '/';
// there is also ->mediumTitle and ->shortTitle
$item['title'] = $video->longTitle;
// in the test, aspect16x9 was the only child of images, not sure whether that is always true
$item['enclosures'] = [
str_replace(self::IMAGEWIDTHPLACEHOLDER, self::IMAGEWIDTH, $video->images->aspect16x9->src)
];
$item['content'] = '<img src="' . $item['enclosures'][0] . '" /><p>';
$item['timestamp'] = $video->broadcastedOn;
$item['uid'] = $video->id;
$item['author'] = $video->publicationService->name;
$this->items[] = $item;
}
$this->title = $processedJSON->title;
date_default_timezone_set($oldTz);
}
/** {@inheritdoc} */
public function getName()
{
if (!empty($this->title)) {
return $this->title;
}
return parent::getName();
}
}

View File

@@ -0,0 +1,63 @@
<?php
class ASRockNewsBridge extends BridgeAbstract
{
const NAME = 'ASRock News Bridge';
const URI = 'https://www.asrock.com';
const DESCRIPTION = 'Returns latest news articles';
const MAINTAINER = 'VerifiedJoseph';
const PARAMETERS = [];
const CACHE_TIMEOUT = 3600; // 1 hour
public function collectData()
{
$html = getSimpleHTMLDOM(self::URI . '/news/index.asp');
$html = defaultLinkTo($html, self::URI . '/news/');
foreach ($html->find('div.inner > a') as $index => $a) {
$item = [];
$articlePath = $a->href;
$articlePageHtml = getSimpleHTMLDOMCached($articlePath, self::CACHE_TIMEOUT);
$articlePageHtml = defaultLinkTo($articlePageHtml, self::URI);
$contents = $articlePageHtml->find('div.Contents', 0);
$item['uri'] = $articlePath;
$item['title'] = $contents->find('h3', 0)->innertext;
$contents->find('h3', 0)->outertext = '';
$item['content'] = $contents->innertext;
$item['timestamp'] = $this->extractDate($a->plaintext);
$img = $a->find('img', 0);
if ($img) {
$item['enclosures'][] = $img->src;
}
$this->items[] = $item;
if (count($this->items) >= 10) {
break;
}
}
}
private function extractDate($text)
{
$dateRegex = '/^([0-9]{4}\/[0-9]{1,2}\/[0-9]{1,2})/';
$text = trim($text);
if (preg_match($dateRegex, $text, $matches)) {
return $matches[1];
}
return '';
}
}

View File

@@ -1,25 +1,37 @@
<?php
class AcrimedBridge extends FeedExpander {
const MAINTAINER = 'qwertygc';
const NAME = 'Acrimed Bridge';
const URI = 'http://www.acrimed.org/';
const CACHE_TIMEOUT = 4800; //2hours
const DESCRIPTION = 'Returns the newest articles';
class AcrimedBridge extends FeedExpander
{
const MAINTAINER = 'qwertygc';
const NAME = 'Acrimed Bridge';
const URI = 'https://www.acrimed.org/';
const CACHE_TIMEOUT = 4800; //2hours
const DESCRIPTION = 'Returns the newest articles';
public function collectData(){
$this->collectExpandableDatas(static::URI . 'spip.php?page=backend');
}
const PARAMETERS = [
[
'limit' => [
'name' => 'limit',
'type' => 'number',
'defaultValue' => -1,
]
]
];
protected function parseItem($newsItem){
$item = parent::parseItem($newsItem);
public function collectData()
{
$url = 'https://www.acrimed.org/spip.php?page=backend';
$limit = $this->getInput('limit');
$this->collectExpandableDatas($url, $limit);
}
$articlePage = getSimpleHTMLDOM($newsItem->link);
$article = sanitize($articlePage->find('article.article1', 0)->innertext);
$article = defaultLinkTo($article, static::URI);
$item['content'] = $article;
return $item;
}
protected function parseItem(array $item)
{
$articlePage = getSimpleHTMLDOM($item['uri']);
$article = sanitize($articlePage->find('article.article1', 0)->innertext);
$article = defaultLinkTo($article, static::URI);
$item['content'] = $article;
return $item;
}
}

View File

@@ -0,0 +1,45 @@
<?php
class ActivisionResearchBridge extends BridgeAbstract
{
const NAME = 'Activision Research Blog';
const URI = 'https://research.activision.com';
const DESCRIPTION = 'Posts from the Activision Research blog';
const MAINTAINER = 'thefranke';
const CACHE_TIMEOUT = 86400; // 24h
public function collectData()
{
$dom = getSimpleHTMLDOM(static::URI);
$dom = $dom->find('div[id="home-blog-feed"]', 0);
if (!$dom) {
throw new \Exception(sprintf('Unable to find css selector on `%s`', $url));
}
$dom = defaultLinkTo($dom, $this->getURI());
foreach ($dom->find('div[class="blog-entry"]') as $article) {
$a = $article->find('a', 0);
$blogimg = extractFromDelimiters($article->find('div[class="blog-img"]', 0)->style, 'url(', ')');
$title = htmlspecialchars_decode($article->find('div[class="title"]', 0)->plaintext);
$author = htmlspecialchars_decode($article->find('div[class="author]', 0)->plaintext);
$date = $article->find('div[class="pubdate"]', 0)->plaintext;
$entry = getSimpleHTMLDOMCached($a->href, static::CACHE_TIMEOUT * 7 * 4);
$entry = defaultLinkTo($entry, $this->getURI());
$content = $entry->find('div[class="blog-body"]', 0);
$tagsremove = ['script', 'iframe', 'input', 'form'];
$content = sanitize($content, $tagsremove);
$content = '<img src="' . static::URI . $blogimg . '" alt="">' . $content;
$this->items[] = [
'title' => $title,
'author' => $author,
'uri' => $a->href,
'content' => $content,
'timestamp' => strtotime($date),
];
}
}
}

View File

@@ -0,0 +1,56 @@
<?php
class AirBreizhBridge extends BridgeAbstract
{
const MAINTAINER = 'fanch317';
const NAME = 'Air Breizh';
const URI = 'https://www.airbreizh.asso.fr/';
const DESCRIPTION = 'Returns newests publications on Air Breizh';
const PARAMETERS = [
'Publications' => [
'theme' => [
'name' => 'Thematique',
'type' => 'list',
'values' => [
'Tout' => '',
'Rapport d\'activite' => 'rapport-dactivite',
'Etude' => 'etudes',
'Information' => 'information',
'Autres documents' => 'autres-documents',
'Plan Régional de Surveillance de la qualité de lair' => 'prsqa',
'Transport' => 'transport'
]
]
]
];
public function getIcon()
{
return 'https://www.airbreizh.asso.fr/voy_content/uploads/2017/11/favicon.png';
}
public function collectData()
{
$html = '';
$html = getSimpleHTMLDOM(static::URI . 'publications/?fwp_publications_thematiques=' . $this->getInput('theme'));
foreach ($html->find('article') as $article) {
$item = [];
// Title
$item['title'] = $article->find('h2', 0)->plaintext;
// Author
$item['author'] = 'Air Breizh';
// Image
$imagelink = $article->find('.card__image', 0)->find('img', 0)->getAttribute('src');
// Content preview
$item['content'] = '<img src="' . $imagelink . '" />
<br/>'
. $article->find('.card__text', 0)->plaintext;
// URL
$item['uri'] = $article->find('.publi__buttons', 0)->find('a', 0)->getAttribute('href');
// ID
$item['id'] = $article->find('.publi__buttons', 0)->find('a', 0)->getAttribute('href');
$this->items[] = $item;
}
}
}

View File

@@ -0,0 +1,76 @@
<?php
class AlbionOnlineBridge extends BridgeAbstract
{
const NAME = 'Albion Online Changelog';
const MAINTAINER = 'otakuf';
const URI = 'https://albiononline.com';
const DESCRIPTION = 'Returns the changes made to the Albion Online';
const CACHE_TIMEOUT = 3600; // 60min
const PARAMETERS = [ [
'postcount' => [
'name' => 'Limit',
'type' => 'number',
'required' => true,
'title' => 'Maximum number of items to return',
'defaultValue' => 5,
],
'language' => [
'name' => 'Language',
'type' => 'list',
'values' => [
'English' => 'en',
'Deutsch' => 'de',
'Polski' => 'pl',
'Français' => 'fr',
'Русский' => 'ru',
'Português' => 'pt',
'Español' => 'es',
],
'title' => 'Language of changelog posts',
'defaultValue' => 'en',
],
'full' => [
'name' => 'Full changelog',
'type' => 'checkbox',
'required' => false,
'title' => 'Enable to receive the full changelog post for each item'
],
]];
public function collectData()
{
$api = 'https://albiononline.com/';
// Example: https://albiononline.com/en/changelog/1/5
$url = $api . $this->getInput('language') . '/changelog/1/' . $this->getInput('postcount');
$html = getSimpleHTMLDOM($url);
foreach ($html->find('li') as $data) {
$item = [];
$item['uri'] = self::URI . $data->find('a', 0)->getAttribute('href');
$item['title'] = trim(explode('|', $data->find('span', 0)->plaintext)[0]);
// Time below work only with en lang. Need to think about solution. May be separate request like getFullChangelog, but to english list for all language
//print_r( date_parse_from_format( 'M j, Y' , 'Sep 9, 2020') );
//$item['timestamp'] = $this->extractDate($a->plaintext);
$item['author'] = 'albiononline.com';
if ($this->getInput('full')) {
$item['content'] = $this->getFullChangelog($item['uri']);
} else {
//$item['content'] = trim(preg_replace('/\s+/', ' ', $data->find('span', 0)->plaintext));
// Just use title, no info at all or use title and date, see above
$item['content'] = $item['title'];
}
$item['uid'] = hash('sha256', $item['title']);
$this->items[] = $item;
}
}
private function getFullChangelog($url)
{
$html = getSimpleHTMLDOMCached($url);
$html = defaultLinkTo($html, self::URI);
return $html->find('div.small-12.columns', 1)->innertext;
}
}

View File

@@ -0,0 +1,87 @@
<?php
class AlfaBankByBridge extends BridgeAbstract
{
const MAINTAINER = 'lassana';
const NAME = 'AlfaBank.by Новости';
const URI = 'https://www.alfabank.by';
const DESCRIPTION = 'Уведомления Alfa-Now — новости от Альфа-Банка';
const CACHE_TIMEOUT = 3600; // 1 hour
const PARAMETERS = [
'News' => [
'business' => [
'name' => 'Альфа Бизнес',
'type' => 'list',
'title' => 'В зависимости от выбора, возращает уведомления для" .
" клиентов физ. лиц либо для клиентов-юридических лиц и ИП',
'values' => [
'Новости' => 'news',
'Новости бизнеса' => 'newsBusiness'
],
'defaultValue' => 'news'
],
'fullContent' => [
'name' => 'Включать содержимое',
'type' => 'checkbox',
'title' => 'Если выбрано, содержимое уведомлений вставляется в поток (работает медленно)'
]
]
];
public function collectData()
{
$business = $this->getInput('business') == 'newsBusiness';
$fullContent = $this->getInput('fullContent') == 'on';
$mainPageUrl = self::URI . '/about/articles/uvedomleniya/';
if ($business) {
$mainPageUrl .= '?business=true';
}
$html = getSimpleHTMLDOM($mainPageUrl);
$limit = 0;
foreach ($html->find('a.notifications__item') as $element) {
if ($limit < 10) {
$item = [];
$item['uid'] = 'urn:sha1:' . hash('sha1', $element->getAttribute('data-notification-id'));
$item['title'] = $element->find('div.item-title', 0)->innertext;
$item['timestamp'] = DateTime::createFromFormat(
'd M Y',
$this->ruMonthsToEn($element->find('div.item-date', 0)->innertext)
)->getTimestamp();
$itemUrl = self::URI . $element->href;
if ($business) {
$itemUrl = str_replace('?business=true', '', $itemUrl);
}
$item['uri'] = $itemUrl;
if ($fullContent) {
$itemHtml = getSimpleHTMLDOM($itemUrl);
if ($itemHtml) {
$item['content'] = $itemHtml->find('div.now-p__content-text', 0)->innertext;
}
}
$this->items[] = $item;
$limit++;
}
}
}
public function getIcon()
{
return static::URI . '/local/images/favicon.ico';
}
private function ruMonthsToEn($date)
{
$ruMonths = [
'Января', 'Февраля', 'Марта', 'Апреля', 'Мая', 'Июня',
'Июля', 'Августа', 'Сентября', 'Октября', 'Ноября', 'Декабря' ];
$enMonths = [
'January', 'February', 'March', 'April', 'May', 'June',
'July', 'August', 'September', 'October', 'November', 'December' ];
return str_replace($ruMonths, $enMonths, $date);
}
}

View File

@@ -0,0 +1,85 @@
<?php
class AllSidesBridge extends BridgeAbstract
{
const NAME = 'AllSides';
const URI = 'https://www.allsides.com';
const DESCRIPTION = 'Balanced news and media bias ratings.';
const MAINTAINER = 'Oliver Nutter';
const PARAMETERS = [
'global' => [
'limit' => [
'name' => 'Number of posts to return',
'type' => 'number',
'defaultValue' => 10,
'required' => false,
'title' => 'Zero or negative values return all posts (ignored if not fetching full article)',
],
'fetch' => [
'name' => 'Fetch full article content',
'type' => 'checkbox',
'defaultValue' => 'checked',
],
],
'Headline Roundups' => [],
];
private const ROUNDUPS_URI = self::URI . '/headline-roundups';
public function collectData()
{
switch ($this->queriedContext) {
case 'Headline Roundups':
$index = getSimpleHTMLDOM(self::ROUNDUPS_URI);
defaultLinkTo($index, self::ROUNDUPS_URI);
$entries = $index->find('table.views-table > tbody > tr');
$limit = (int) $this->getInput('limit');
$fetch = (bool) $this->getInput('fetch');
if ($limit > 0 && $fetch) {
$entries = array_slice($entries, 0, $limit);
}
foreach ($entries as $entry) {
$item = [
'title' => $entry->find('.views-field-name', 0)->text(),
'uri' => $entry->find('a', 0)->href,
'timestamp' => $entry->find('.date-display-single', 0)->content,
'author' => 'AllSides Staff',
];
if ($fetch) {
$article = getSimpleHTMLDOMCached($item['uri']);
defaultLinkTo($article, $item['uri']);
$item['content'] = $article->find('.story-id-page-description', 0);
foreach ($article->find('.page-tags a') as $tag) {
$item['categories'][] = $tag->text();
}
}
$this->items[] = $item;
}
break;
}
}
public function getName()
{
if ($this->queriedContext) {
return self::NAME . " - {$this->queriedContext}";
}
return self::NAME;
}
public function getURI()
{
switch ($this->queriedContext) {
case 'Headline Roundups':
return self::ROUNDUPS_URI;
}
return self::URI;
}
}

120
bridges/AllegroBridge.php Normal file
View File

@@ -0,0 +1,120 @@
<?php
class AllegroBridge extends BridgeAbstract
{
const NAME = 'Allegro';
const URI = 'https://www.allegro.pl';
const DESCRIPTION = 'Returns the search results from the Allegro.pl shopping and bidding portal';
const MAINTAINER = 'wrobelda';
const PARAMETERS = [[
'url' => [
'name' => 'Search URL',
'title' => 'Copy the URL from your browser\'s address bar after searching for your items and paste it here',
'exampleValue' => 'https://allegro.pl/kategoria/swieze-warzywa-cebula-318660',
'required' => true,
],
'cookie' => [
'name' => 'The complete cookie value',
'title' => 'Paste the cookie value from your browser, otherwise 403 gets returned',
'required' => true,
],
'includeSponsoredOffers' => [
'type' => 'checkbox',
'name' => 'Include Sponsored Offers',
'defaultValue' => 'checked'
],
'includePromotedOffers' => [
'type' => 'checkbox',
'name' => 'Include Promoted Offers',
'defaultValue' => 'checked'
]
]];
public function getName()
{
$url = $this->getInput('url');
if (!$url) {
return parent::getName();
}
$parsedUrl = parse_url($url, PHP_URL_QUERY);
if (!$parsedUrl) {
return parent::getName();
}
parse_str($parsedUrl, $fields);
if (array_key_exists('string', $fields)) {
$f = urldecode($fields['string']);
} else {
$f = false;
}
if ($f) {
return $f;
}
return parent::getName();
}
public function getURI()
{
return $this->getInput('url') ?? parent::getURI();
}
public function collectData()
{
# make sure we order by the most recently listed offers
$url = preg_replace('/([?&])order=[^&]+(&|$)/', '$1', $this->getInput('url'));
$url .= (parse_url($url, PHP_URL_QUERY) ? '&' : '?') . 'order=n';
$html = getContents($url, [], [CURLOPT_COOKIE => $this->getInput('cookie')]);
$storeData = null;
if (preg_match('/<script[^>]*>\s*(\{\s*?"__listing_StoreState".*\})\s*<\/script>/i', $html, $match)) {
$data = json_decode($match[1], true);
$storeData = $data['__listing_StoreState'] ?? null;
}
foreach ($storeData['items']['elements'] as $elements) {
if (!array_key_exists('offerId', $elements)) {
continue;
}
if (!$this->getInput('includeSponsoredOffers') && $elements['isSponsored']) {
continue;
}
if (!$this->getInput('includePromotedOffers') && $elements['promoted']) {
continue;
}
$item = [];
$item['uid'] = $elements['offerId'];
$item['uri'] = $elements['url'];
$item['title'] = $elements['alt'];
$image = $elements['photos'][0]['medium'];
if ($image) {
$item['enclosures'] = [$image . '#.image'];
}
$price = $elements['price']['mainPrice']['amount'];
$currency = $elements['price']['mainPrice']['currency'];
$sellerType = $elements['seller']['title'];
$item['categories'] = [$sellerType];
$description = '';
foreach ($elements['parameters'] as $parameter) {
$item['categories'] = array_merge($item['categories'], $parameter['values']);
$description .= '<dt>' . $parameter['name'] . ': ' . implode(',', $parameter['values']) . '</dt>';
}
$item['content'] = '<div><strong>'
. $price . ' ' . $currency
. '</strong></div><dl><dt>'
. $sellerType . '</dt>'
. $description
. '</dl><hr>';
$this->items[] = $item;
}
}
}

View File

@@ -1,87 +1,107 @@
<?php
class AllocineFRBridge extends BridgeAbstract {
const MAINTAINER = 'superbaillot.net';
const NAME = 'Allo Cine Bridge';
const CACHE_TIMEOUT = 25200; // 7h
const URI = 'http://www.allocine.fr/';
const DESCRIPTION = 'Bridge for allocine.fr';
const PARAMETERS = array( array(
'category' => array(
'name' => 'category',
'type' => 'list',
'required' => true,
'exampleValue' => 'Faux Raccord',
'title' => 'Select your category',
'values' => array(
'Faux Raccord' => 'faux-raccord',
'Top 5' => 'top-5',
'Tueurs en Séries' => 'tueurs-en-serie'
)
)
));
class AllocineFRBridge extends BridgeAbstract
{
const MAINTAINER = 'superbaillot.net';
const NAME = 'Allo Cine Bridge';
const CACHE_TIMEOUT = 25200; // 7h
const URI = 'https://www.allocine.fr';
const DESCRIPTION = 'Bridge for allocine.fr';
const PARAMETERS = [ [
'category' => [
'name' => 'Emission',
'type' => 'list',
'title' => 'Sélectionner l\'emission',
'values' => [
'Faux Raccord' => 'faux-raccord',
'Fanzone' => 'fanzone',
'Game In Ciné' => 'game-in-cine',
'Pour la faire courte' => 'pour-la-faire-courte',
'Home Cinéma' => 'home-cinema',
'PILS - Par Ici Les Sorties' => 'pils-par-ici-les-sorties',
'AlloCiné : l\'émission, sur LeStream' => 'allocine-lemission-sur-lestream',
'Give Me Five' => 'give-me-five',
'Aviez-vous remarqué ?' => 'aviez-vous-remarque',
'Et paf, il est mort' => 'et-paf-il-est-mort',
'The Big Fan Theory' => 'the-big-fan-theory',
'Clichés' => 'cliches',
'Complètement...' => 'completement',
'#Fun Facts' => 'fun-facts',
'Origin Story' => 'origin-story',
]
]
]];
public function getURI(){
if(!is_null($this->getInput('category'))) {
public function getURI()
{
if (!is_null($this->getInput('category'))) {
$categories = [
'faux-raccord' => '/video/programme-12284/',
'fanzone' => '/video/programme-12298/',
'game-in-cine' => '/video/programme-12288/',
'pour-la-faire-courte' => '/video/programme-20960/',
'home-cinema' => '/video/programme-12287/',
'pils-par-ici-les-sorties' => '/video/programme-25789/',
'allocine-lemission-sur-lestream' => '/video/programme-25123/',
'give-me-five' => '/video/programme-21919/saison-34518/',
'aviez-vous-remarque' => '/video/programme-19518/',
'et-paf-il-est-mort' => '/video/programme-25113/',
'the-big-fan-theory' => '/video/programme-20403/',
'cliches' => '/video/programme-24834/',
'completement' => '/video/programme-23859/',
'fun-facts' => '/video/programme-23040/',
'origin-story' => '/video/programme-25667/'
];
switch($this->getInput('category')) {
case 'faux-raccord':
$uri = static::URI . 'video/programme-12284/saison-32180/';
break;
case 'top-5':
$uri = static::URI . 'video/programme-12299/saison-29561/';
break;
case 'tueurs-en-serie':
$uri = static::URI . 'video/programme-12286/saison-22938/';
break;
}
$category = $this->getInput('category');
if (array_key_exists($category, $categories)) {
return static::URI . $this->getLastSeasonURI($categories[$category]);
} else {
throwClientException('Emission inconnue');
}
}
return $uri;
}
return parent::getURI();
}
return parent::getURI();
}
private function getLastSeasonURI($category)
{
$html = getSimpleHTMLDOMCached(static::URI . $category, 86400);
$seasonLink = $html->find('section[class=section-wrap section]', 0)->find('div[class=cf]', 0)->find('a', 0);
$URI = $seasonLink->href;
return $URI;
}
public function getName(){
if(!is_null($this->getInput('category'))) {
return self::NAME . ' : '
.array_search(
$this->getInput('category'),
self::PARAMETERS[$this->queriedContext]['category']['values']
);
}
public function getName()
{
if (!is_null($this->getInput('category'))) {
return self::NAME . ' : ' . $this->getKey('category');
}
return parent::getName();
}
return parent::getName();
}
public function collectData(){
public function collectData()
{
$html = getSimpleHTMLDOM($this->getURI());
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Could not request ' . $this->getURI() . ' !');
foreach ($html->find('div[class=gd-col-left]', 0)->find('div[class*=video-card]') as $element) {
$item = [];
$category = array_search(
$this->getInput('category'),
self::PARAMETERS[$this->queriedContext]['category']['values']
);
$title = $element->find('a[class*=meta-title-link]', 0);
$content = trim(defaultLinkTo($element->outertext, static::URI));
foreach($html->find('.media-meta-list figure.media-meta-fig') as $element) {
$item = array();
// Replace image 'src' with the one in 'data-src'
$content = preg_replace('@src="data:image/gif;base64,[A-Za-z0-9+\/]*"@', '', $content);
$content = preg_replace('@data-src=@', 'src=', $content);
$title = $element->find('div.titlebar h3.title a', 0);
$content = trim($element->innertext);
$figCaption = strpos($content, $category);
if($figCaption !== false) {
$content = str_replace('src="/', 'src="' . static::URI, $content);
$content = str_replace('href="/', 'href="' . static::URI, $content);
$content = str_replace('src=\'/', 'src=\'' . static::URI, $content);
$content = str_replace('href=\'/', 'href=\'' . static::URI, $content);
$item['content'] = $content;
$item['title'] = trim($title->innertext);
$item['uri'] = static::URI . $title->href;
$this->items[] = $item;
}
}
}
// Remove date in the content to prevent content update while the video is getting older
$content = preg_replace('@<div class="meta-sub light">.*<span>[^<]*</span>[^<]*</div>@', '', $content);
$item['content'] = $content;
$item['title'] = trim($title->innertext);
$item['uri'] = static::URI . '/' . substr($title->href, 1);
$this->items[] = $item;
}
}
}

View File

@@ -0,0 +1,66 @@
<?php
class AllocineFRSortiesBridge extends BridgeAbstract
{
const MAINTAINER = 'Simounet';
const NAME = 'AlloCiné Sorties Bridge';
const CACHE_TIMEOUT = 25200; // 7h
const BASE_URI = 'https://www.allocine.fr';
const URI = self::BASE_URI . '/film/sorties-semaine/';
const DESCRIPTION = 'Bridge for AlloCiné - Sorties cinéma cette semaine';
public function getName()
{
return self::NAME;
}
public function collectData()
{
$html = getSimpleHTMLDOM($this->getURI());
foreach ($html->find('section.section.section-wrap', 0)->find('li.mdl') as $element) {
$item = [];
$thumb = $element->find('figure.thumbnail', 0);
$meta = $element->find('div.meta-body', 0);
$synopsis = $element->find('div.synopsis', 0);
$date = $element->find('span.date', 0);
$title = $element->find('a[class*=meta-title-link]', 0);
$content = trim(defaultLinkTo($thumb->outertext . $meta->outertext . $synopsis->outertext, static::URI));
// Replace image 'src' with the one in 'data-src'
$content = preg_replace('@src="data:image/gif;base64,[A-Za-z0-9=+\/]*"@', '', $content);
$content = preg_replace('@data-src=@', 'src=', $content);
$item['content'] = $content;
$item['title'] = trim($title->innertext);
$item['timestamp'] = $this->frenchPubDateToTimestamp($date->plaintext);
$item['uri'] = static::BASE_URI . '/' . substr($title->href, 1);
$this->items[] = $item;
}
}
private function frenchPubDateToTimestamp($date)
{
return strtotime(
strtr(
strtolower($date),
[
'janvier' => 'jan',
'février' => 'feb',
'mars' => 'march',
'avril' => 'apr',
'mai' => 'may',
'juin' => 'jun',
'juillet' => 'jul',
'août' => 'aug',
'septembre' => 'sep',
'octobre' => 'oct',
'novembre' => 'nov',
'décembre' => 'dec'
]
)
);
}
}

View File

@@ -1,94 +1,105 @@
<?php
class AmazonBridge extends BridgeAbstract {
class AmazonBridge extends BridgeAbstract
{
const MAINTAINER = 'Alexis CHEMEL';
const NAME = 'Amazon';
const URI = 'https://www.amazon.com/';
const CACHE_TIMEOUT = 3600; // 1h
const DESCRIPTION = 'Returns products from Amazon search';
const MAINTAINER = 'Alexis CHEMEL';
const NAME = 'Amazon';
const URI = 'https://www.amazon.com/';
const CACHE_TIMEOUT = 3600; // 1h
const DESCRIPTION = 'Returns products from Amazon search';
const PARAMETERS = [[
'q' => [
'name' => 'Keyword',
'required' => true,
'exampleValue' => 'watch',
],
'sort' => [
'name' => 'Sort by',
'type' => 'list',
'values' => [
'Relevance' => 'relevanceblender',
'Price: Low to High' => 'price-asc-rank',
'Price: High to Low' => 'price-desc-rank',
'Average Customer Review' => 'review-rank',
'Newest Arrivals' => 'date-desc-rank',
],
'defaultValue' => 'relevanceblender',
],
'tld' => [
'name' => 'Country',
'type' => 'list',
'values' => [
'Australia' => 'com.au',
'Brazil' => 'com.br',
'Canada' => 'ca',
'China' => 'cn',
'France' => 'fr',
'Germany' => 'de',
'India' => 'in',
'Italy' => 'it',
'Japan' => 'co.jp',
'Mexico' => 'com.mx',
'Netherlands' => 'nl',
'Poland' => 'pl',
'Spain' => 'es',
'Sweden' => 'se',
'Turkey' => 'com.tr',
'United Kingdom' => 'co.uk',
'United States' => 'com',
],
'defaultValue' => 'com',
],
]];
const PARAMETERS = array(array(
'q' => array(
'name' => 'Keyword',
'required' => true,
),
'sort' => array(
'name' => 'Sort by',
'type' => 'list',
'required' => false,
'values' => array(
'Relevance' => 'relevanceblender',
'Price: Low to High' => 'price-asc-rank',
'Price: High to Low' => 'price-desc-rank',
'Average Customer Review' => 'review-rank',
'Newest Arrivals' => 'date-desc-rank',
),
'defaultValue' => 'relevanceblender',
),
'tld' => array(
'name' => 'Country',
'type' => 'list',
'required' => true,
'values' => array(
'Australia' => 'com.au',
'Brazil' => 'com.br',
'Canada' => 'ca',
'China' => 'cn',
'France' => 'fr',
'Germany' => 'de',
'India' => 'in',
'Italy' => 'it',
'Japan' => 'co.jp',
'Mexico' => 'com.mx',
'Netherlands' => 'nl',
'Spain' => 'es',
'United Kingdom' => 'co.uk',
'United States' => 'com',
),
'defaultValue' => 'com',
),
));
public function collectData()
{
$baseUrl = sprintf('https://www.amazon.%s', $this->getInput('tld'));
public function getName(){
if(!is_null($this->getInput('tld')) && !is_null($this->getInput('q'))) {
return 'Amazon.'.$this->getInput('tld').': '.$this->getInput('q');
}
$url = sprintf(
'%s/s/?field-keywords=%s&sort=%s',
$baseUrl,
urlencode($this->getInput('q')),
$this->getInput('sort')
);
return parent::getName();
}
$dom = getSimpleHTMLDOM($url);
public function collectData() {
$elements = $dom->find('div.s-result-item');
$uri = 'https://www.amazon.'.$this->getInput('tld').'/';
$uri .= 's/?field-keywords='.urlencode($this->getInput('q')).'&sort='.$this->getInput('sort');
foreach ($elements as $element) {
$item = [];
$html = getSimpleHTMLDOM($uri)
or returnServerError('Could not request Amazon.');
$title = $element->find('h2', 0);
if (!$title) {
continue;
}
foreach($html->find('li.s-result-item') as $element) {
$item['title'] = $title->innertext;
$item = array();
$itemUrl = $element->find('a', 0)->href;
$item['uri'] = urljoin($baseUrl, $itemUrl);
// Title
$title = $element->find('h2', 0);
$image = $element->find('img', 0);
if ($image) {
$item['content'] = '<img src="' . $image->getAttribute('src') . '" /><br />';
}
$item['title'] = html_entity_decode($title->innertext, ENT_QUOTES);
$price = $element->find('span.a-price > .a-offscreen', 0);
if ($price) {
$item['content'] .= $price->innertext;
}
// Url
$uri = $title->parent()->getAttribute('href');
$uri = substr($uri, 0, strrpos($uri, '/'));
$this->items[] = $item;
}
}
$item['uri'] = substr($uri, 0, strrpos($uri, '/'));
public function getName()
{
if (!is_null($this->getInput('tld')) && !is_null($this->getInput('q'))) {
return 'Amazon.' . $this->getInput('tld') . ': ' . $this->getInput('q');
}
// Content
$image = $element->find('img', 0);
$price = $element->find('span.s-price', 0);
$price = ($price) ? $price->innertext : '';
$item['content'] = '<img src="'.$image->getAttribute('src').'" /><br />'.$price;
$this->items[] = $item;
}
}
return parent::getName();
}
}

View File

@@ -0,0 +1,269 @@
<?php
class AmazonPriceTrackerBridge extends BridgeAbstract
{
const MAINTAINER = 'captn3m0, sal0max, bagnacauda';
const NAME = 'Amazon Price Tracker';
const URI = 'https://www.amazon.com/';
const CACHE_TIMEOUT = 3600; // 1h
const DESCRIPTION = 'Tracks price for a single product on Amazon';
const PARAMETERS = [
[
'asin' => [
'name' => 'ASIN',
'required' => true,
'exampleValue' => 'B0923XT6K7',
// https://stackoverflow.com/a/12827734
'pattern' => 'B[\dA-Z]{9}|\d{9}(X|\d)',
],
'tld' => [
'name' => 'Country',
'type' => 'list',
'values' => [
'Australia' => 'com.au',
'Brazil' => 'com.br',
'Canada' => 'ca',
'China' => 'cn',
'France' => 'fr',
'Germany' => 'de',
'India' => 'in',
'Italy' => 'it',
'Japan' => 'co.jp',
'Mexico' => 'com.mx',
'Netherlands' => 'nl',
'Poland' => 'pl',
'Spain' => 'es',
'Sweden' => 'se',
'Turkey' => 'com.tr',
'United Kingdom' => 'co.uk',
'United States' => 'com',
],
'defaultValue' => 'com',
],
]];
const PRICE_SELECTORS = [
'#priceblock_ourprice',
'.priceBlockBuyingPriceString',
'#newBuyBoxPrice',
'#tp_price_block_total_price_ww',
'span.offer-price',
'.a-color-price',
];
const WHITESPACE = " \t\n\r\0\x0B\xC2\xA0";
protected $title;
/**
* Generates domain name given a amazon TLD
*/
private function getDomainName()
{
return 'https://www.amazon.' . $this->getInput('tld');
}
/**
* Generates URI for a Amazon product page
*/
public function getURI()
{
if (!is_null($this->getInput('asin'))) {
return $this->getDomainName() . '/dp/' . $this->getInput('asin');
}
return parent::getURI();
}
/**
* Scrapes the product title from the html page
* returns the default title if scraping fails
*/
private function getTitle($html)
{
$titleTag = $html->find('#productTitle', 0);
if (!$titleTag) {
return $this->getDefaultTitle();
} else {
return trim(html_entity_decode($titleTag->innertext, ENT_QUOTES));
}
}
/**
* Title used by the feed if none could be found
*/
private function getDefaultTitle()
{
return 'Amazon.' . $this->getInput('tld') . ': ' . $this->getInput('asin');
}
/**
* Returns name for the feed
* Uses title (already scraped) if it has one
*/
public function getName()
{
if (isset($this->title)) {
return $this->title;
} else {
return parent::getName();
}
}
private function parseDynamicImage($attribute)
{
$json = json_decode(html_entity_decode($attribute), true);
if ($json and count($json) > 0) {
return array_keys($json)[0];
}
}
/**
* Returns a generated image tag for the product
*/
private function getImage($html)
{
$image = 'https://placekitten.com/200/300';
$imageSrc = $html->find('#main-image-container img', 0);
if ($imageSrc) {
$hiresImage = $imageSrc->getAttribute('data-old-hires');
$dynamicImageAttribute = $imageSrc->getAttribute('data-a-dynamic-image');
$image = $hiresImage ?: $this->parseDynamicImage($dynamicImageAttribute);
}
return <<<EOT
<img width="300" style="max-width:300;max-height:300" src="$image" alt="{$this->title}" />
EOT;
}
/**
* Return \simple_html_dom object
* for the entire html of the product page
*/
private function getHtml()
{
$uri = $this->getURI();
return getSimpleHTMLDOM($uri);
}
private function scrapePriceFromMetrics($html)
{
$asinData = $html->find('#cerberus-data-metrics', 0);
// <div id="cerberus-data-metrics" style="display: none;"
// data-asin="B00WTHJ5SU" data-asin-price="14.99" data-asin-shipping="0"
// data-asin-currency-code="USD" data-substitute-count="-1" ... />
if ($asinData) {
return [
'price' => $asinData->getAttribute('data-asin-price'),
'currency' => $asinData->getAttribute('data-asin-currency-code'),
'shipping' => $asinData->getAttribute('data-asin-shipping')
];
}
return false;
}
private function scrapePriceTwister($html)
{
$json = $html->find('.twister-plus-buying-options-price-data', 0);
if ($json == null) {
return null;
}
$data = json_decode($json->innertext, true);
foreach ($data as $key => $value) {
$value = $value[0];
return [
'displayPrice' => $value['displayPrice'],
'price' => $value['priceAmount'],
'currency' => $value['currencySymbol'],
'shipping' => null,
];
}
return null;
}
private function scrapePriceGeneric($html)
{
$default = [
'price' => null,
'displayPrice' => null,
'currency' => null,
'shipping' => null,
];
$priceDiv = null;
foreach (self::PRICE_SELECTORS as $sel) {
$priceDiv = $html->find($sel, 0);
if ($priceDiv) {
break;
}
}
if (!$priceDiv) {
return $default;
}
$priceString = str_replace(str_split(self::WHITESPACE), '', $priceDiv->plaintext);
$price = null;
$priceFound = false;
// find longest repeated string
for ($offset = 0; $offset < strlen($priceString); $offset++) {
for ($length = 1; substr_count($priceString, substr($priceString, $offset, $length + 1)) >= 2; $length++) {
$priceFound = true;
}
if ($priceFound) {
$price = substr($priceString, $offset, $length);
break;
}
}
$currency = str_replace($price, '', $priceString);
if ($price != null && $currency != null) {
return [
'price' => $price,
'displayPrice' => null,
'currency' => $currency,
'shipping' => null
];
}
return $default;
}
public function collectData()
{
$html = $this->getHtml();
$this->title = $this->getTitle($html);
$image = $this->getImage($html);
$data = $this->scrapePriceTwister($html) ?? $this->scrapePriceGeneric($html);
// render
$content = '';
$price = $data['displayPrice'];
if (!$price) {
$price = sprintf('%s %s', $data['price'], $data['currency']);
}
$content .= sprintf('%s<br>Price: %s', $image, $price);
if ($data['shipping'] !== null) {
$content .= sprintf('<br>Shipping: %s %s</br>', $data['shipping'], $data['currency']);
}
$item = [
'title' => $this->title,
'uri' => $this->getURI(),
'content' => $content,
// This is to ensure that feed readers notice the price change
'uid' => md5($data['price'])
];
$this->items[] = $item;
}
}

278
bridges/AnfrBridge.php Normal file
View File

@@ -0,0 +1,278 @@
<?php
class AnfrBridge extends BridgeAbstract
{
const NAME = 'ANFR';
const URI = 'https://data.anfr.fr/';
const DESCRIPTION = 'Fetches data from the French administration "Agence Nationale des Fréquences".';
const CACHE_TIMEOUT = 604800; // 7d
const MAINTAINER = 'quent1';
const PARAMETERS = [
'Données sur les réseaux mobiles' => [
'departement' => [
'name' => 'Département',
'type' => 'list',
'values' => [
'Tous' => null,
'Ain' => '001',
'Aisne' => '002',
'Allier' => '003',
'Alpes-de-Haute-Provence' => '004',
'Hautes-Alpes' => '005',
'Alpes-Maritimes' => '006',
'Ardèche' => '007',
'Ardennes' => '008',
'Ariège' => '009',
'Aube' => '010',
'Aude' => '011',
'Aveyron' => '012',
'Bouches-du-Rhône' => '013',
'Calvados' => '014',
'Cantal' => '015',
'Charente' => '016',
'Charente-Maritime' => '017',
'Cher' => '018',
'Corrèze' => '019',
'Corse-du-Sud' => '02A',
'Haute-Corse' => '02B',
'Côte-d\'Or' => '021',
'Côtes-d\'Armor' => '022',
'Creuse' => '023',
'Dordogne' => '024',
'Doubs' => '025',
'Drôme' => '026',
'Eure' => '027',
'Eure-et-Loir' => '028',
'Finistère' => '029',
'Gard' => '030',
'Haute-Garonne' => '031',
'Gers' => '032',
'Gironde' => '033',
'Hérault' => '034',
'Ille-et-Vilaine' => '035',
'Indre' => '036',
'Indre-et-Loire' => '037',
'Isère' => '038',
'Jura' => '039',
'Landes' => '040',
'Loir-et-Cher' => '041',
'Loire' => '042',
'Haute-Loire' => '043',
'Loire-Atlantique' => '044',
'Loiret' => '045',
'Lot' => '046',
'Lot-et-Garonne' => '047',
'Lozère' => '048',
'Maine-et-Loire' => '049',
'Manche' => '050',
'Marne' => '051',
'Haute-Marne' => '052',
'Mayenne' => '053',
'Meurthe-et-Moselle' => '054',
'Meuse' => '055',
'Morbihan' => '056',
'Moselle' => '057',
'Nièvre' => '058',
'Nord' => '059',
'Oise' => '060',
'Orne' => '061',
'Pas-de-Calais' => '062',
'Puy-de-Dôme' => '063',
'Pyrénées-Atlantiques' => '064',
'Hautes-Pyrénées' => '065',
'Pyrénées-Orientales' => '066',
'Bas-Rhin' => '067',
'Haut-Rhin' => '068',
'Rhône' => '069',
'Haute-Saône' => '070',
'Saône-et-Loire' => '071',
'Sarthe' => '072',
'Savoie' => '073',
'Haute-Savoie' => '074',
'Paris' => '075',
'Seine-Maritime' => '076',
'Seine-et-Marne' => '077',
'Yvelines' => '078',
'Deux-Sèvres' => '079',
'Somme' => '080',
'Tarn' => '081',
'Tarn-et-Garonne' => '082',
'Var' => '083',
'Vaucluse' => '084',
'Vendée' => '085',
'Vienne' => '086',
'Haute-Vienne' => '087',
'Vosges' => '088',
'Yonne' => '089',
'Territoire de Belfort' => '090',
'Essonne' => '091',
'Hauts-de-Seine' => '092',
'Seine-Saint-Denis' => '093',
'Val-de-Marne' => '094',
'Val-d\'Oise' => '095',
'Guadeloupe' => '971',
'Martinique' => '972',
'Guyane' => '973',
'La Réunion' => '974',
'Saint-Pierre-et-Miquelon' => '975',
'Mayotte' => '976',
'Saint-Barthélemy' => '977',
'Saint-Martin' => '978',
'Terres australes et antarctiques françaises' => '984',
'Wallis-et-Futuna' => '986',
'Polynésie française' => '987',
'Nouvelle-Calédonie' => '988',
'Île de Clipperton' => '989'
]
],
'generation' => [
'name' => 'Génération',
'type' => 'list',
'values' => [
'Tous' => null,
'2G' => '2G',
'3G' => '3G',
'4G' => '4G',
'5G' => '5G',
]
],
'operateur' => [
'name' => 'Opérateur',
'type' => 'list',
'values' => [
'Tous' => null,
'Bouygues Télécom' => 'BOUYGUES TELECOM',
'Dauphin Télécom' => 'DAUPHIN TELECOM',
'Digiciel' => 'DIGICEL',
'Free Caraïbes' => 'FREE CARAIBES',
'Free Mobile' => 'FREE MOBILE',
'GLOBALTEL' => 'GLOBALTEL',
'Office des postes et télécommunications de Nouvelle Calédonie' => 'Gouv Nelle Calédonie (OPT)',
'Maore Mobile' => 'MAORE MOBILE',
'ONATi' => 'ONATI',
'Orange' => 'ORANGE',
'Outremer Telecom' => 'OUTREMER TELECOM',
'Vodafone polynésie' => 'PMT/VODAPHONE',
'SFR' => 'SFR',
'SPM Télécom' => 'SPM TELECOM',
'Service des Postes et Télécommunications de Polynésie Française' => 'Gouv Nelle Calédonie (OPT)',
'SRR' => 'SRR',
'Station étrangère' => 'Station étrangère',
'Telco OI' => 'TELCO IO',
'United Telecommunication Services Caraïbes' => 'UTS Caraibes',
'Ora Mobile' => 'VITI SAS',
'Zeop' => 'ZEOP'
]
],
'statut' => [
'name' => 'Statut',
'type' => 'list',
'values' => [
'Tous' => null,
'En service' => 'En service',
'Projet approuvé' => 'Projet approuvé',
'Techniquement opérationnel' => 'Techniquement opérationnel',
]
]
]
];
public function collectData()
{
$urlParts = [
'id' => 'observatoire_2g_3g_4g',
'resource_id' => '88ef0887-6b0f-4d3f-8545-6d64c8f597da',
'fields' => 'id,adm_lb_nom,sta_nm_dpt,emr_lb_systeme,generation,date_maj,sta_nm_anfr,adr_lb_lieu,adr_lb_add1,adr_lb_add2,adr_lb_add3,adr_nm_cp,statut',
'rows' => 10000
];
if (!empty($this->getInput('departement'))) {
$urlParts['refine.sta_nm_dpt'] = urlencode($this->getInput('departement'));
}
if (!empty($this->getInput('generation'))) {
$urlParts['refine.generation'] = $this->getInput('generation');
}
if (!empty($this->getInput('operateur'))) {
// http_build_query() already does urlencoding so this call is redundant
$urlParts['refine.adm_lb_nom'] = urlencode($this->getInput('operateur'));
}
if (!empty($this->getInput('statut'))) {
$urlParts['refine.statut'] = urlencode($this->getInput('statut'));
}
// API seems to not play well with urlencoded data
$url = urljoin(static::URI, '/d4c/api/records/1.0/download/?' . urldecode(http_build_query($urlParts)));
$json = getContents($url);
$data = Json::decode($json, false);
$records = $data->records;
$frequenciesByStation = [];
foreach ($records as $record) {
if (!isset($frequenciesByStation[$record->fields->sta_nm_anfr])) {
$street = sprintf(
'%s %s %s',
$record->fields->adr_lb_add1 ?? '',
$record->fields->adr_lb_add2 ?? '',
$record->fields->adr_lb_add3 ?? ''
);
$frequenciesByStation[$record->fields->sta_nm_anfr] = [
'id' => $record->fields->sta_nm_anfr,
'operator' => $record->fields->adm_lb_nom,
'frequencies' => [],
'lastUpdate' => 0,
'address' => [
'street' => trim($street),
'postCode' => $record->fields->adr_nm_cp,
'city' => $record->fields->adr_lb_lieu
]
];
}
$frequenciesByStation[$record->fields->sta_nm_anfr]['frequencies'][] = [
'generation' => $record->fields->generation,
'frequency' => $record->fields->emr_lb_systeme,
'status' => $record->fields->statut,
'updatedAt' => strtotime($record->fields->date_maj),
];
$frequenciesByStation[$record->fields->sta_nm_anfr]['lastUpdate'] = max(
$frequenciesByStation[$record->fields->sta_nm_anfr]['lastUpdate'],
strtotime($record->fields->date_maj)
);
}
usort($frequenciesByStation, static fn ($a, $b) => $b['lastUpdate'] <=> $a['lastUpdate']);
foreach ($frequenciesByStation as $station) {
$title = sprintf(
'[%s] Mise à jour de la station n°%s à %s (%s)',
$station['operator'],
$station['id'],
$station['address']['city'],
$station['address']['postCode']
);
$array_reduce = array_reduce($station['frequencies'], static function ($carry, $frequency) {
return sprintf('%s<li>%s : %s</li>', $carry, $frequency['frequency'], $frequency['status']);
}, '');
$content = sprintf(
'<h1>Adresse complète</h1><p>%s<br>%s<br>%s</p><h1>Fréquences</h1><p><ul>%s</ul></p>',
$station['address']['street'],
$station['address']['postCode'],
$station['address']['city'],
$array_reduce
);
$this->items[] = [
'uid' => $station['id'],
'timestamp' => $station['lastUpdate'],
'title' => $title,
'content' => $content,
];
}
}
}

218
bridges/AnidexBridge.php Normal file
View File

@@ -0,0 +1,218 @@
<?php
class AnidexBridge extends BridgeAbstract
{
const MAINTAINER = 'ORelio';
const NAME = 'Anidex';
const URI = 'http://anidex.info/'; // anidex.info has ddos-guard so we need to use anidex.moe
const ALTERNATE_URI = 'https://anidex.moe/'; // anidex.moe returns 301 unless Host is set to anidex.info
const ALTERNATE_HOST = 'anidex.info'; // Correct host for requesting anidex.moe without 301 redirect
const DESCRIPTION = 'Returns the newest torrents, with optional search criteria.';
const PARAMETERS = [
[
'id' => [
'name' => 'Category',
'type' => 'list',
'values' => [
'All categories' => '0',
'Anime' => '1,2,3',
'Anime - Sub' => '1',
'Anime - Raw' => '2',
'Anime - Dub' => '3',
'Live Action' => '4,5',
'Live Action - Sub' => '4',
'Live Action - Raw' => '5',
'Light Novel' => '6',
'Manga' => '7,8',
'Manga - Translated' => '7',
'Manga - Raw' => '8',
'Music' => '9,10,11',
'Music - Lossy' => '9',
'Music - Lossless' => '10',
'Music - Video' => '11',
'Games' => '12',
'Applications' => '13',
'Pictures' => '14',
'Adult Video' => '15',
'Other' => '16'
]
],
'lang_id' => [
'name' => 'Language',
'type' => 'list',
'values' => [
'All languages' => '0',
'English' => '1',
'Japanese' => '2',
'Polish' => '3',
'Serbo-Croatian' => '4',
'Dutch' => '5',
'Italian' => '6',
'Russian' => '7',
'German' => '8',
'Hungarian' => '9',
'French' => '10',
'Finnish' => '11',
'Vietnamese' => '12',
'Greek' => '13',
'Bulgarian' => '14',
'Spanish (Spain)' => '15',
'Portuguese (Brazil)' => '16',
'Portuguese (Portugal)' => '17',
'Swedish' => '18',
'Arabic' => '19',
'Danish' => '20',
'Chinese (Simplified)' => '21',
'Bengali' => '22',
'Romanian' => '23',
'Czech' => '24',
'Mongolian' => '25',
'Turkish' => '26',
'Indonesian' => '27',
'Korean' => '28',
'Spanish (LATAM)' => '29',
'Persian' => '30',
'Malaysian' => '31'
]
],
'group_id' => [
'name' => 'Group ID',
'type' => 'number'
],
'r' => [
'name' => 'Hide Remakes',
'type' => 'checkbox'
],
'b' => [
'name' => 'Only Batches',
'type' => 'checkbox'
],
'a' => [
'name' => 'Only Authorized',
'type' => 'checkbox'
],
'q' => [
'name' => 'Keyword',
'description' => 'Keyword(s)',
'type' => 'text'
],
'h' => [
'name' => 'Adult content',
'type' => 'list',
'values' => [
'No filter' => '0',
'Hide +18' => '1',
'Only +18' => '2'
]
]
]
];
public function collectData()
{
// Build Search URL from user-provided parameters
$search_url = self::ALTERNATE_URI . '?s=upload_timestamp&o=desc';
foreach (['id', 'lang_id', 'group_id'] as $param_name) {
$param = $this->getInput($param_name);
if (!empty($param) && intval($param) != 0 && ctype_digit(str_replace(',', '', $param))) {
$search_url .= '&' . $param_name . '=' . $param;
}
}
foreach (['r', 'b', 'a'] as $param_name) {
$param = $this->getInput($param_name);
if (!empty($param) && boolval($param)) {
$search_url .= '&' . $param_name . '=1';
}
}
$query = $this->getInput('q');
if (!empty($query)) {
$search_url .= '&q=' . urlencode($query);
}
$opt = [];
$h = $this->getInput('h');
if (!empty($h) && intval($h) != 0 && ctype_digit($h)) {
$opt[CURLOPT_COOKIE] = 'anidex_h_toggle=' . $h;
}
// We need to use a different Host HTTP header to reach the correct page on ALTERNATE_URI
$headers = ['Host: ' . self::ALTERNATE_HOST];
// The HTTPS certificate presented by anidex.moe is for anidex.info. We need to ignore this.
// As a consequence, the bridge is intentionally marked as insecure by setting self::URI to http://
$opt[CURLOPT_SSL_VERIFYHOST] = 0;
$opt[CURLOPT_SSL_VERIFYPEER] = 0;
// Retrieve torrent listing from search results, which does not contain torrent description
$html = getSimpleHTMLDOM($search_url, $headers, $opt);
$links = $html->find('a');
$results = [];
foreach ($links as $link) {
if (strpos($link->href, '/torrent/') === 0 && !in_array($link->href, $results)) {
$results[] = $link->href;
}
}
if (empty($results) && empty($this->getInput('q'))) {
throwServerException('No results from Anidex: ' . $search_url);
}
//Process each item individually
foreach ($results as $element) {
//Limit total amount of requests
if (count($this->items) >= 20) {
break;
}
$torrent_id = str_replace('/torrent/', '', $element);
//Ignore entries without valid torrent ID
if ($torrent_id != 0 && ctype_digit($torrent_id)) {
//Retrieve data for this torrent ID
$item_browse_uri = self::URI . 'torrent/' . $torrent_id;
$item_fetch_uri = self::ALTERNATE_URI . 'torrent/' . $torrent_id;
//Retrieve full description from torrent page (cached for 24 hours: 86400 seconds)
if ($item_html = getSimpleHTMLDOMCached($item_fetch_uri, 86400, $headers, $opt)) {
//Retrieve data from page contents
$item_title = str_replace(' (Torrent) - AniDex ', '', $item_html->find('title', 0)->plaintext);
$item_desc = $item_html->find('div.panel-body', 0);
$item_author = trim($item_html->find('span.fa-user', 0)->parent()->plaintext);
$item_date = strtotime(trim($item_html->find('span.fa-clock', 0)->parent()->plaintext));
$item_image = $this->getURI() . 'images/user_logos/default.png';
//Check for description-less torrent andn optionally extract image
$desc_title_found = false;
foreach ($item_html->find('h3.panel-title') as $h3) {
if (strpos($h3, 'Description') !== false) {
$desc_title_found = true;
break;
}
}
if ($desc_title_found) {
//Retrieve image for thumbnail or generic logo fallback
foreach ($item_desc->find('img') as $img) {
if (strpos($img->src, 'prez') === false) {
$item_image = $img->src;
break;
}
}
$item_desc = trim($item_desc->innertext);
} else {
$item_desc = '<em>No description.</em>';
}
//Build and add final item
$item = [];
$item['uri'] = $item_browse_uri;
$item['title'] = $item_title;
$item['author'] = $item_author;
$item['timestamp'] = $item_date;
$item['enclosures'] = [$item_image];
$item['content'] = $item_desc;
$this->items[] = $item;
}
}
$element = null;
}
$results = null;
}
}

View File

@@ -1,135 +1,133 @@
<?php
class AnimeUltimeBridge extends BridgeAbstract {
const MAINTAINER = 'ORelio';
const NAME = 'Anime-Ultime';
const URI = 'http://www.anime-ultime.net/';
const CACHE_TIMEOUT = 10800; // 3h
const DESCRIPTION = 'Returns the 10 newest releases posted on Anime-Ultime';
const PARAMETERS = array( array(
'type' => array(
'name' => 'Type',
'type' => 'list',
'values' => array(
'Everything' => '',
'Anime' => 'A',
'Drama' => 'D',
'Tokusatsu' => 'T'
)
)
));
class AnimeUltimeBridge extends BridgeAbstract
{
const MAINTAINER = 'ORelio';
const NAME = 'Anime-Ultime';
const URI = 'http://www.anime-ultime.net/';
const CACHE_TIMEOUT = 10800; // 3h
const DESCRIPTION = 'Returns the newest releases posted on Anime-Ultime.';
const PARAMETERS = [ [
'type' => [
'name' => 'Type',
'type' => 'list',
'values' => [
'Everything' => '',
'Anime' => 'A',
'Drama' => 'D',
'Tokusatsu' => 'T'
]
]
]];
private $filter = 'Releases';
private $filter = 'Releases';
public function collectData(){
public function collectData()
{
//Add type filter if provided
$typeFilter = $this->getKey('type');
//Add type filter if provided
$typeFilter = array_search(
$this->getInput('type'),
self::PARAMETERS[$this->queriedContext]['type']['values']
);
//Build date and filters for making requests
$thismonth = date('mY') . $typeFilter;
$lastmonth = date('mY', mktime(0, 0, 0, date('n') - 1, 1, date('Y'))) . $typeFilter;
//Build date and filters for making requests
$thismonth = date('mY') . $typeFilter;
$lastmonth = date('mY', mktime(0, 0, 0, date('n') - 1, 1, date('Y'))) . $typeFilter;
//Process each HTML page until having 10 releases
$processedOK = 0;
foreach ([$thismonth, $lastmonth] as $requestFilter) {
$url = self::URI . 'history-0-1/' . $requestFilter;
$html = getContents($url);
// Convert html from iso-8859-1 => utf8
$html = utf8_encode($html);
$html = str_get_html($html);
//Process each HTML page until having 10 releases
$processedOK = 0;
foreach (array($thismonth, $lastmonth) as $requestFilter) {
//Relases are sorted by day : process each day individually
foreach ($html->find('div.history', 0)->find('h3') as $daySection) {
//Retrieve day and build date information
$dateString = $daySection->plaintext;
$day = intval(substr($dateString, strpos($dateString, ' ') + 1, 2));
$item_date = strtotime(str_pad($day, 2, '0', STR_PAD_LEFT)
. '-'
. substr($requestFilter, 0, 2)
. '-'
. substr($requestFilter, 2, 4));
//Retrive page contents
$url = self::URI . 'history-0-1/' . $requestFilter;
$html = getSimpleHTMLDOM($url)
or returnServerError('Could not request Anime-Ultime: ' . $url);
//<h3>day</h3><br /><table><tr> <-- useful data in table rows
$release = $daySection->next_sibling()->next_sibling()->first_child();
//Relases are sorted by day : process each day individually
foreach($html->find('div.history', 0)->find('h3') as $daySection) {
//Process each release of that day, ignoring first table row: contains table headers
while (!is_null($release = $release->next_sibling())) {
if (count($release->find('td')) > 0) {
//Retrieve metadata from table columns
$item_link_element = $release->find('td', 0)->find('a', 0);
$item_uri = self::URI . $item_link_element->href;
$item_name = html_entity_decode($item_link_element->plaintext);
//Retrieve day and build date information
$dateString = $daySection->plaintext;
$day = intval(substr($dateString, strpos($dateString, ' ') + 1, 2));
$item_date = strtotime(str_pad($day, 2, '0', STR_PAD_LEFT)
. '-'
. substr($requestFilter, 0, 2)
. '-'
. substr($requestFilter, 2, 4));
$item_image = self::URI . substr(
$item_link_element->onmouseover,
37,
strpos($item_link_element->onmouseover, ' ', 37) - 37
);
//<h3>day</h3><br /><table><tr> <-- useful data in table rows
$release = $daySection->next_sibling()->next_sibling()->first_child();
$item_episode = html_entity_decode(
str_pad(
$release->find('td', 1)->plaintext,
2,
'0',
STR_PAD_LEFT
)
);
//Process each release of that day, ignoring first table row: contains table headers
while(!is_null($release = $release->next_sibling())) {
if(count($release->find('td')) > 0) {
$item_fansub = $release->find('td', 2)->plaintext;
$item_type = $release->find('td', 4)->plaintext;
//Retrieve metadata from table columns
$item_link_element = $release->find('td', 0)->find('a', 0);
$item_uri = self::URI . $item_link_element->href;
$item_name = html_entity_decode($item_link_element->plaintext);
$item_episode = html_entity_decode(
str_pad(
$release->find('td', 1)->plaintext,
2,
'0',
STR_PAD_LEFT
)
);
if (!empty($item_uri)) {
// Retrieve description from description page
$html_item = getContents($item_uri);
// Convert html from iso-8859-1 => utf8
$html_item = utf8_encode($html_item);
$item_description = substr(
$html_item,
strpos($html_item, 'class="principal_contain" align="center">') + 41
);
$item_description = substr(
$item_description,
0,
strpos($item_description, '<div id="table">')
);
$item_fansub = $release->find('td', 2)->plaintext;
$item_type = $release->find('td', 4)->plaintext;
// Convert relative image src into absolute image src, remove line breaks
$item_description = defaultLinkTo($item_description, self::URI);
$item_description = str_replace("\r", '', $item_description);
$item_description = str_replace("\n", '', $item_description);
if(!empty($item_uri)) {
//Build and add final item
$item = [];
$item['uri'] = $item_uri;
$item['title'] = $item_name . ' ' . $item_type . ' ' . $item_episode;
$item['author'] = $item_fansub;
$item['timestamp'] = $item_date;
$item['enclosures'] = [$item_image];
$item['content'] = $item_description;
$this->items[] = $item;
$processedOK++;
// Retrieve description from description page and
// convert relative image src info absolute image src
$html_item = getContents($item_uri)
or returnServerError('Could not request Anime-Ultime: ' . $item_uri);
$item_description = substr(
$html_item,
strpos($html_item, 'class="principal_contain" align="center">') + 41
);
$item_description = substr($item_description,
0,
strpos($item_description, '<div id="table">')
);
$item_description = str_replace(
'src="images', 'src="' . self::URI . 'images',
$item_description
);
$item_description = str_replace("\r", '', $item_description);
$item_description = str_replace("\n", '', $item_description);
$item_description = utf8_encode($item_description);
//Stop processing once limit is reached
if ($processedOK >= 10) {
return;
}
}
}
}
}
}
}
//Build and add final item
$item = array();
$item['uri'] = $item_uri;
$item['title'] = $item_name . ' ' . $item_type . ' ' . $item_episode;
$item['author'] = $item_fansub;
$item['timestamp'] = $item_date;
$item['content'] = $item_description;
$this->items[] = $item;
$processedOK++;
//Stop processing once limit is reached
if ($processedOK >= 10)
return;
}
}
}
}
}
}
public function getName() {
if(!is_null($this->getInput('type'))) {
$typeFilter = array_search(
$this->getInput('type'),
self::PARAMETERS[$this->queriedContext]['type']['values']
);
return 'Latest ' . $typeFilter . ' - Anime-Ultime Bridge';
}
return parent::getName();
}
public function getName()
{
if (!is_null($this->getInput('type'))) {
return 'Latest ' . $this->getKey('type') . ' - Anime-Ultime Bridge';
}
return parent::getName();
}
}

View File

@@ -0,0 +1,87 @@
<?php
class AnisearchBridge extends BridgeAbstract
{
const MAINTAINER = 'Tone866';
const NAME = 'Anisearch';
const URI = 'https://www.anisearch.de';
const CACHE_TIMEOUT = 1800; // 30min
const DESCRIPTION = 'Feed for Anisearch';
const PARAMETERS = [[
'category' => [
'name' => 'Dub',
'type' => 'list',
'values' => [
'DE'
=> 'https://www.anisearch.de/anime/index/page-1?char=all&synchro=de&sort=date&order=desc&view=4',
'EN'
=> 'https://www.anisearch.de/anime/index/page-1?char=all&synchro=en&sort=date&order=desc&view=4',
'JP'
=> 'https://www.anisearch.de/anime/index/page-1?char=all&synchro=ja&sort=date&order=desc&view=4'
]
],
'trailers' => [
'name' => 'Trailers',
'type' => 'checkbox',
'title' => 'Will include trailes',
'defaultValue' => false
]
]];
public function collectData()
{
$baseurl = 'https://www.anisearch.de/';
$trailers = false;
$trailers = $this->getInput('trailers');
$limit = 10;
if ($trailers) {
$limit = 5;
}
$dom = getSimpleHTMLDOM($this->getInput('category'));
foreach ($dom->find('li.btype0') as $key => $li) {
if ($key >= $limit) {
break;
}
$a = $li->find('a', 0);
$title = $a->find('span.title', 0);
$url = $baseurl . $a->href;
//get article
$domarticle = getSimpleHTMLDOM($url);
$content = $domarticle->find('div.details-text', 0);
//get header-image and set absolute src
$headerimage = $domarticle->find('img#details-cover', 0);
$src = $headerimage->src;
foreach ($content->find('.hidden') as $element) {
$element->remove();
}
//get trailer
$ytlink = '';
if ($trailers) {
$trailerlink = $domarticle->find('section#trailers > div > div.swiper > ul.swiper-wrapper > li.swiper-slide > a', 0);
if (isset($trailerlink)) {
$trailersite = getSimpleHTMLDOM($baseurl . $trailerlink->href);
$trailer = $trailersite->find('div#video > iframe', 0);
$trailer = $trailer->{'data-xsrc'};
$ytlink = <<<EOT
<br /><iframe width="560" height="315" src="$trailer" title="YouTube video player"
frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
EOT;
}
}
$this->items[] = [
'title' => $title->plaintext,
'uri' => $url,
'content' => $headerimage . '<br />' . $content . $ytlink
];
}
}
}

View File

@@ -0,0 +1,183 @@
<?php
class AnnasArchiveBridge extends BridgeAbstract
{
const NAME = 'Anna\'s Archive';
const MAINTAINER = 'phantop';
const URI = 'https://annas-archive.org/';
const DESCRIPTION = 'Returns books from Anna\'s Archive';
const PARAMETERS = [
[
'q' => [
'name' => 'Query',
'exampleValue' => 'apothecary diaries',
'required' => true,
],
'ext' => [
'name' => 'Extension',
'type' => 'list',
'values' => [
'Any' => null,
'azw3' => 'azw3',
'cbr' => 'cbr',
'cbz' => 'cbz',
'djvu' => 'djvu',
'epub' => 'epub',
'fb2' => 'fb2',
'fb2.zip' => 'fb2.zip',
'mobi' => 'mobi',
'pdf' => 'pdf',
]
],
'lang' => [
'name' => 'Language',
'type' => 'list',
'values' => [
'Any' => null,
'Afrikaans [af]' => 'af',
'Arabic [ar]' => 'ar',
'Bangla [bn]' => 'bn',
'Belarusian [be]' => 'be',
'Bulgarian [bg]' => 'bg',
'Catalan [ca]' => 'ca',
'Chinese [zh]' => 'zh',
'Church Slavic [cu]' => 'cu',
'Croatian [hr]' => 'hr',
'Czech [cs]' => 'cs',
'Danish [da]' => 'da',
'Dongxiang [sce]' => 'sce',
'Dutch [nl]' => 'nl',
'English [en]' => 'en',
'French [fr]' => 'fr',
'German [de]' => 'de',
'Greek [el]' => 'el',
'Hebrew [he]' => 'he',
'Hindi [hi]' => 'hi',
'Hungarian [hu]' => 'hu',
'Indonesian [id]' => 'id',
'Irish [ga]' => 'ga',
'Italian [it]' => 'it',
'Japanese [ja]' => 'ja',
'Kazakh [kk]' => 'kk',
'Korean [ko]' => 'ko',
'Latin [la]' => 'la',
'Latvian [lv]' => 'lv',
'Lithuanian [lt]' => 'lt',
'Luxembourgish [lb]' => 'lb',
'Ndolo [ndl]' => 'ndl',
'Norwegian [no]' => 'no',
'Persian [fa]' => 'fa',
'Polish [pl]' => 'pl',
'Portuguese [pt]' => 'pt',
'Romanian [ro]' => 'ro',
'Russian [ru]' => 'ru',
'Serbian [sr]' => 'sr',
'Spanish [es]' => 'es',
'Swedish [sv]' => 'sv',
'Tamil [ta]' => 'ta',
'Traditional Chinese [zhHant]' => 'zhHant',
'Turkish [tr]' => 'tr',
'Ukrainian [uk]' => 'uk',
'Unknown language' => '_empty',
'Unknown language [und]' => 'und',
'Unknown language [urdu]' => 'urdu',
'Urdu [ur]' => 'ur',
'Vietnamese [vi]' => 'vi',
'Welsh [cy]' => 'cy',
]
],
'content' => [
'name' => 'Type',
'type' => 'list',
'values' => [
'Any' => null,
'Book (fiction)' => 'book_fiction',
'Book (nonfiction)' => 'book_nonfiction',
'Book (unknown)' => 'book_unknown',
'Comic book' => 'book_comic',
'Journal article' => 'journal_article',
'Magazine' => 'magazine',
'Standards document' => 'standards_document',
]
],
'src' => [
'name' => 'Source',
'type' => 'list',
'values' => [
'Any' => null,
'Internet Archive' => 'ia',
'Libgen.li' => 'lgli',
'Libgen.rs' => 'lgrs',
'SciHub' => 'scihub',
'ZLibrary' => 'zlib',
]
],
]
];
public function collectData()
{
$url = $this->getURI();
$list = getSimpleHTMLDOMCached($url);
$list = defaultLinkTo($list, self::URI);
// Don't attempt to do anything if not found message is given
if ($list->find('.js-not-found-additional')) {
return;
}
$elements = $list->find('#aarecord-list > div');
foreach ($elements as $element) {
// stop added entries once partial match list starts
if (str_contains($element->innertext, 'partial match')) {
break;
}
if ($element = $element->find('a', 0)) {
$item = [];
$item['title'] = $element->find('h3', 0)->plaintext;
$item['author'] = $element->find('div.italic', 0)->plaintext;
$item['uri'] = $element->href;
$item['content'] = $element->plaintext;
$item['uid'] = $item['uri'];
$item_html = getSimpleHTMLDOMCached($item['uri'], 86400 * 20);
if ($item_html) {
$item_html = defaultLinkTo($item_html, self::URI);
$item['content'] .= $item_html->find('main img', 0);
$item['content'] .= $item_html->find('main .mt-4', 0); // Summary
foreach ($item_html->find('main ul.mb-4 > li > a.js-download-link') as $file) {
if (!str_contains($file->href, 'fast_download')) {
$item['enclosures'][] = $file->href;
}
}
// Remove bulk torrents from enclosures list
$item['enclosures'] = array_diff($item['enclosures'], [self::URI . 'datasets']);
}
$this->items[] = $item;
}
}
}
public function getName()
{
$name = parent::getName();
if ($this->getInput('q') != null) {
$name .= ' - ' . $this->getInput('q');
}
return $name;
}
public function getURI()
{
$params = array_filter([ // Filter to remove non-provided parameters
'q' => $this->getInput('q'),
'ext' => $this->getInput('ext'),
'lang' => $this->getInput('lang'),
'src' => $this->getInput('src'),
'content' => $this->getInput('content'),
]);
$url = parent::getURI() . 'search?sort=newest&' . http_build_query($params);
return $url;
}
}

View File

@@ -0,0 +1,234 @@
<?php
class AppleAppStoreBridge extends BridgeAbstract
{
const MAINTAINER = 'captn3m0';
const NAME = 'Apple App Store';
const URI = 'https://apps.apple.com/';
const CACHE_TIMEOUT = 3600; // 1h
const DESCRIPTION = 'Returns version updates for a specific application';
const PARAMETERS = [[
'id' => [
'name' => 'Application ID',
'required' => true,
'exampleValue' => '310633997'
],
'p' => [
'name' => 'Platform',
'type' => 'list',
'values' => [
'iPad' => 'ipad',
'iPhone' => 'iphone',
'Mac' => 'mac',
// The following 2 are present in responses
// but not yet tested
'Web' => 'web',
'Apple TV' => 'appletv',
],
'defaultValue' => 'iphone',
],
'country' => [
'name' => 'Store Country',
'type' => 'list',
'values' => [
'US' => 'US',
'India' => 'IN',
'Canada' => 'CA',
'Germany' => 'DE',
'Netherlands' => 'NL',
'Belgium (NL)' => 'BENL',
'Belgium (FR)' => 'BEFR',
'France' => 'FR',
'Italy' => 'IT',
'United Kingdom' => 'UK',
'Spain' => 'ES',
'Portugal' => 'PT',
'Australia' => 'AU',
'New Zealand' => 'NZ',
'Indonesia' => 'ID',
'Brazil' => 'BR',
],
'defaultValue' => 'US',
],
'debug' => [
'name' => 'Debug Mode',
'type' => 'checkbox',
'defaultValue' => false
]
]];
const PLATFORM_MAPPING = [
'iphone' => 'ios',
'ipad' => 'ios',
'mac' => 'osx'
];
private $name;
private function makeHtmlUrl()
{
$id = $this->getInput('id');
$country = $this->getInput('country');
return sprintf('https://apps.apple.com/%s/app/id%s', $country, $id);
}
private function makeJsonUrl()
{
$id = $this->getInput('id');
$country = $this->getInput('country');
$platform = $this->getInput('p');
$platform_param = ($platform === 'mac') ? 'mac' : $platform;
return sprintf(
'https://amp-api-edge.apps.apple.com/v1/catalog/%s/apps/%s?platform=%s&extend=versionHistory',
$country,
$id,
$platform_param
);
}
public function getName()
{
if (isset($this->name)) {
return sprintf('%s - AppStore Updates', $this->name);
}
return parent::getName();
}
private function debugLog($message)
{
if ($this->getInput('debug')) {
$this->logger->info(sprintf('[AppleAppStoreBridge] %s', $message));
}
}
private function getHtml()
{
$url = $this->makeHtmlUrl();
$this->debugLog(sprintf('Fetching HTML from: %s', $url));
return getSimpleHTMLDOM($url);
}
private function getJWTToken()
{
$html = $this->getHtml();
$meta = $html->find('meta[name="web-experience-app/config/environment"]', 0);
if (!$meta || !isset($meta->content)) {
throw new \Exception('JWT token not found in page content');
}
$decoded_content = urldecode($meta->content);
$this->debugLog('Found meta tag content');
try {
$decoded_json = Json::decode($decoded_content);
} catch (\Exception $e) {
throw new \Exception(sprintf('Failed to parse JSON from meta tag: %s', $e->getMessage()));
}
if (!isset($decoded_json['MEDIA_API']['token'])) {
throw new \Exception('Token field not found in JSON structure');
}
$token = $decoded_json['MEDIA_API']['token'];
$this->debugLog('Successfully extracted JWT token');
return $token;
}
private function getAppData()
{
$token = $this->getJWTToken();
$url = $this->makeJsonUrl();
$this->debugLog(sprintf('Fetching data from API: %s', $url));
$headers = [
'Authorization: Bearer ' . $token,
'Origin: https://apps.apple.com',
'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
];
$content = getContents($url, $headers);
try {
$json = Json::decode($content);
} catch (\Exception $e) {
throw new \Exception(sprintf('Failed to parse API response: %s', $e->getMessage()));
}
if (!isset($json['data']) || empty($json['data'])) {
throw new \Exception('No app data found in API response');
}
$this->debugLog('Successfully retrieved app data from API');
return $json['data'][0];
}
private function extractAppDetails($data)
{
if (isset($data['attributes'])) {
$this->name = $data['attributes']['name'] ?? null;
$author = $data['attributes']['artistName'] ?? null;
$this->debugLog(sprintf('Found app details in attributes: %s by %s', $this->name, $author));
return [$this->name, $author];
}
// Fallback to default values if not found
$this->name = sprintf('App %s', $this->getInput('id'));
$this->debugLog(sprintf('App details not found, using default: %s', $this->name));
return [$this->name, 'Unknown Developer'];
}
private function getVersionHistory($data)
{
$platform = $this->getInput('p');
$this->debugLog(sprintf('Extracting version history for platform: %s', $platform));
// Get the mapped platform key (ios for iPhone/iPad, osx for Mac)
$platform_key = self::PLATFORM_MAPPING[$platform] ?? $platform;
$version_history = $data['attributes']['platformAttributes'][$platform_key]['versionHistory'] ?? [];
if (empty($version_history)) {
$this->debugLog(sprintf('No version history found for %s', $platform));
}
return $version_history;
}
public function collectData()
{
$this->debugLog(sprintf('Getting data for %s app', $this->getInput('p')));
$data = $this->getAppData();
// Get app name and author using array destructuring
[$name, $author] = $this->extractAppDetails($data);
// Get version history
$version_history = $this->getVersionHistory($data);
$this->debugLog(sprintf('Found %d versions for %s', count($version_history), $name));
foreach ($version_history as $entry) {
$version = $entry['versionDisplay'] ?? 'Unknown Version';
$release_notes = $entry['releaseNotes'] ?? 'No release notes available';
$release_date = $entry['releaseDate'] ?? 'Unknown Date';
$item = [];
$item['title'] = sprintf('%s - %s', $name, $version);
$item['content'] = nl2br($release_notes) ?: 'No release notes available';
$item['timestamp'] = $release_date;
$item['author'] = $author;
$item['uri'] = $this->makeHtmlUrl();
$this->items[] = $item;
}
$this->debugLog(sprintf('Successfully collected %d items', count($this->items)));
}
}

View File

@@ -0,0 +1,117 @@
<?php
class AppleMusicBridge extends BridgeAbstract
{
const NAME = 'Apple Music';
const URI = 'https://www.apple.com';
const DESCRIPTION = 'Fetches the latest releases from an artist';
const MAINTAINER = 'bockiii';
const PARAMETERS = [[
'artist' => [
'name' => 'Artist ID',
'exampleValue' => '909253',
'required' => true,
],
'limit' => [
'name' => 'Latest X Releases (max 50)',
'defaultValue' => '10',
'required' => true,
],
]];
const CACHE_TIMEOUT = 60 * 60 * 6; // 6 hours
private $title;
public function collectData()
{
$items = $this->getJson();
$artist = $this->getArtist($items);
$this->title = $artist->artistName;
foreach ($items as $item) {
if ($item->wrapperType === 'collection') {
$copyright = $item->copyright ?? '';
$artworkUrl500 = str_replace('/100x100', '/500x500', $item->artworkUrl100);
$artworkUrl2000 = str_replace('/100x100', '/2000x2000', $item->artworkUrl100);
$escapedCollectionName = htmlspecialchars($item->collectionName);
$this->items[] = [
'title' => $item->collectionName,
'uri' => $item->collectionViewUrl,
'timestamp' => $item->releaseDate,
'enclosures' => $artworkUrl500,
'author' => $item->artistName,
'content' => "<figure>
<img srcset=\"$item->artworkUrl60 60w, $item->artworkUrl100 100w, $artworkUrl500 500w, $artworkUrl2000 2000w\"
sizes=\"100%\" src=\"$artworkUrl2000\"
alt=\"Cover of $escapedCollectionName\"
style=\"display: block; margin: 0 auto;\" />
<figcaption>
from <a href=\"$artist->artistLinkUrl\">$item->artistName</a><br />$copyright
</figcaption>
</figure>",
];
}
}
}
private function getJson()
{
# Limit the amount of releases to 50
if ($this->getInput('limit') > 50) {
$limit = 50;
} else {
$limit = $this->getInput('limit');
}
$url = 'https://itunes.apple.com/lookup?id=' . $this->getInput('artist') . '&entity=album&limit=' . $limit . '&sort=recent';
$html = getSimpleHTMLDOM($url);
$json = json_decode($html);
$result = $json->results;
if (!is_array($result) || count($result) == 0) {
throwServerException('There is no artist with id "' . $this->getInput('artist') . '".');
}
return $result;
}
private function getArtist($json)
{
$nameArray = array_filter($json, function ($obj) {
return $obj->wrapperType == 'artist';
});
if (count($nameArray) === 1) {
return $nameArray[0];
}
return parent::getName();
}
public function getName()
{
if (isset($this->title)) {
return $this->title;
}
return parent::getName();
}
public function getIcon()
{
if (empty($this->getInput('artist'))) {
return parent::getIcon();
}
// it isn't necessary to set the correct artist name into the url
$url = 'https://music.apple.com/us/artist/jon-bellion/' . $this->getInput('artist');
$html = getSimpleHTMLDOMCached($url);
$image = $html->find('meta[property="og:image"]', 0)->content;
$imageUpdatedSize = preg_replace('/\/\d*x\d*cw/i', '/144x144-999', $image);
return $imageUpdatedSize;
}
}

View File

@@ -0,0 +1,118 @@
<?php
class ArsTechnicaBridge extends FeedExpander
{
const MAINTAINER = 'phantop';
const NAME = 'Ars Technica';
const URI = 'https://arstechnica.com/';
const DESCRIPTION = 'Returns the latest articles from Ars Technica';
const PARAMETERS = [[
'section' => [
'name' => 'Site section',
'type' => 'list',
'defaultValue' => 'index',
'values' => [
'All' => 'index',
'Apple' => 'apple',
'Board Games' => 'cardboard',
'Cars' => 'cars',
'Features' => 'features',
'Gaming' => 'gaming',
'Information Technology' => 'technology-lab',
'Science' => 'science',
'Staff Blogs' => 'staff-blogs',
'Tech Policy' => 'tech-policy',
'Tech' => 'gadgets',
]
]
]];
public function collectData()
{
$url = 'https://feeds.arstechnica.com/arstechnica/' . $this->getInput('section');
$this->collectExpandableDatas($url, 10);
}
protected function parseItem(array $item)
{
$item_html = getSimpleHTMLDOMCached($item['uri']);
$item_html = defaultLinkTo($item_html, self::URI);
$content = '';
$header = $item_html->find('article header', 0);
$leading = $header->find('p[class*=leading]', 0);
if ($leading != null) {
$content .= '<p>' . $leading->innertext . '</p>';
}
$intro_image = $header->find('img.intro-image', 0);
if ($intro_image != null) {
$content .= '<figure>' . $intro_image;
$image_caption = $header->find('.caption .caption-content', 0);
if ($image_caption != null) {
$content .= '<figcaption>' . $image_caption->innertext . '</figcaption>';
}
$content .= '</figure>';
}
foreach ($item_html->find('.post-content') as $content_tag) {
$content .= $content_tag->innertext;
}
$item['content'] = str_get_html($content);
$parsely = $item_html->find('[name="parsely-page"]', 0);
$parsely_json = json_decode(html_entity_decode($parsely->content), true);
$item['categories'] = $parsely_json['tags'];
// Some lightboxes are nested in figures. I'd guess that's a
// bug in the website
foreach ($item['content']->find('figure div div.ars-lightbox') as $weird_lightbox) {
$weird_lightbox->parent->parent->outertext = $weird_lightbox;
}
// It's easier to reconstruct the whole thing than remove
// duplicate reactive tags
foreach ($item['content']->find('.ars-lightbox') as $lightbox) {
$lightbox_content = '';
foreach ($lightbox->find('.ars-lightbox-item') as $lightbox_item) {
$img = $lightbox_item->find('img', 0);
if ($img != null) {
$lightbox_content .= '<figure>' . $img;
$caption = $lightbox_item->find('div.pswp-caption-content', 0);
if ($caption != null) {
$credit = $lightbox_item->find('div.ars-gallery-caption-credit', 0);
if ($credit != null) {
$credit->innertext = 'Credit: ' . $credit->innertext;
}
$lightbox_content .= '<figcaption>' . $caption->innertext . '</figcaption>';
}
$lightbox_content .= '</figure>';
}
}
$lightbox->innertext = $lightbox_content;
}
// remove various ars advertising
foreach ($item['content']->find('.ars-interlude-container') as $ad) {
$ad->remove();
}
foreach ($item['content']->find('.toc-container') as $toc) {
$toc->remove();
}
// Mostly YouTube videos
$iframes = $item['content']->find('iframe');
foreach ($iframes as $iframe) {
$iframe->outertext = '<a href="' . $iframe->src . '">' . $iframe->src . '</a>';
}
// This fixed padding around the former iframes and actual inline videos
foreach ($item['content']->find('div[style*=aspect-ratio]') as $styled) {
$styled->removeAttribute('style');
}
$item['content'] = backgroundToImg($item['content']);
$item['uid'] = strval($parsely_json['post_id']);
return $item;
}
}

View File

@@ -0,0 +1,101 @@
<?php
class ArtStationBridge extends BridgeAbstract
{
const NAME = 'ArtStation';
const URI = 'https://www.artstation.com';
const DESCRIPTION = 'Fetches the latest ten artworks from a search query on ArtStation.';
const MAINTAINER = 'thefranke';
const CACHE_TIMEOUT = 3600; // 1h
const PARAMETERS = [
'Search Query' => [
'q' => [
'name' => 'Search term',
'required' => true,
'exampleValue' => 'bird'
]
]
];
public function getIcon()
{
return 'https://www.artstation.com/assets/favicon-58653022bc38c1905ac7aa1b10bffa6b.ico';
}
public function getName()
{
return self::NAME . ': ' . $this->getInput('q');
}
private function fetchSearch($searchQuery)
{
$data = '{"query":"' . $searchQuery . '","page":1,"per_page":50,"sorting":"date",';
$data .= '"pro_first":"1","filters":[],"additional_fields":[]}';
$header = [
'Content-Type: application/json',
'Accept: application/json'
];
$opts = [
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => $data,
CURLOPT_RETURNTRANSFER => true
];
$jsonSearchURL = self::URI . '/api/v2/search/projects.json';
$jsonSearchStr = getContents($jsonSearchURL, $header, $opts);
return json_decode($jsonSearchStr);
}
private function fetchProject($hashID)
{
$jsonProjectURL = self::URI . '/projects/' . $hashID . '.json';
$jsonProjectStr = getContents($jsonProjectURL);
return json_decode($jsonProjectStr);
}
public function collectData()
{
$searchTerm = $this->getInput('q');
$jsonQuery = $this->fetchSearch($searchTerm);
foreach ($jsonQuery->data as $media) {
// get detailed info about media item
$jsonProject = $this->fetchProject($media->hash_id);
// create item
$item = [];
$item['title'] = $media->title;
$item['uri'] = $media->url;
$item['timestamp'] = strtotime($jsonProject->published_at);
$item['author'] = $media->user->full_name;
$item['categories'] = implode(',', $jsonProject->tags);
$item['content'] = '<a href="'
. $media->url
. '"><img style="max-width: 100%" src="'
. $jsonProject->cover_url
. '"></a><p>'
. $jsonProject->description
. '</p>';
$numAssets = count($jsonProject->assets);
if ($numAssets > 1) {
$item['content'] .= '<p><a href="'
. $media->url
. '">Project contains '
. ($numAssets - 1)
. ' more item(s).</a></p>';
}
$this->items[] = $item;
if (count($this->items) >= 10) {
break;
}
}
}
}

View File

@@ -1,102 +1,166 @@
<?php
class Arte7Bridge extends BridgeAbstract {
const MAINTAINER = 'mitsukarenai';
const NAME = 'Arte +7';
const URI = 'https://www.arte.tv/';
const CACHE_TIMEOUT = 1800; // 30min
const DESCRIPTION = 'Returns newest videos from ARTE +7';
class Arte7Bridge extends BridgeAbstract
{
const NAME = 'Arte +7';
const URI = 'https://www.arte.tv/';
const CACHE_TIMEOUT = 1800; // 30min
const DESCRIPTION = 'Returns newest videos from ARTE +7';
const API_TOKEN = 'Nzc1Yjc1ZjJkYjk1NWFhN2I2MWEwMmRlMzAzNjI5NmU3NWU3ODg4ODJjOWMxNTMxYzEzZGRjYjg2ZGE4MmIwOA';
const API_TOKEN = 'Nzc1Yjc1ZjJkYjk1NWFhN2I2MWEwMmRlMzAzNjI5NmU3NWU3ODg4ODJjOWMxNTMxYzEzZGRjYjg2ZGE4MmIwOA';
const PARAMETERS = array(
'Catégorie (Français)' => array(
'catfr' => array(
'type' => 'list',
'name' => 'Catégorie',
'values' => array(
'Toutes les vidéos (français)' => null,
'Actu & société' => 'ACT',
'Séries & fiction' => 'SER',
'Cinéma' => 'CIN',
'Arts & spectacles classiques' => 'ARS',
'Culture pop' => 'CPO',
'Découverte' => 'DEC',
'Histoire' => 'HIST',
'Science' => 'SCI',
'Autre' => 'AUT'
)
)
),
'Catégorie (Allemand)' => array(
'catde' => array(
'type' => 'list',
'name' => 'Catégorie',
'values' => array(
'Alle Videos (deutsch)' => null,
'Aktuelles & Gesellschaft' => 'ACT',
'Fernsehfilme & Serien' => 'SER',
'Kino' => 'CIN',
'Kunst & Kultur' => 'ARS',
'Popkultur & Alternativ' => 'CPO',
'Entdeckung' => 'DEC',
'Geschichte' => 'HIST',
'Wissenschaft' => 'SCI',
'Sonstiges' => 'AUT'
)
)
)
);
const PARAMETERS = [
'global' => [
'sort_by' => [
'type' => 'list',
'name' => 'Sort by',
'required' => false,
'defaultValue' => null,
'values' => [
'Default' => null,
'Video rights start date' => 'videoRightsBegin',
'Video rights end date' => 'videoRightsEnd',
'Brodcast date' => 'broadcastBegin',
'Creation date' => 'creationDate',
'Last modified' => 'lastModified',
'Number of views' => 'views',
'Number of views per period' => 'viewsPeriod',
'Available screens' => 'availableScreens',
'Episode' => 'episode'
],
],
'sort_direction' => [
'type' => 'list',
'name' => 'Sort direction',
'required' => false,
'defaultValue' => 'DESC',
'values' => [
'Ascending' => 'ASC',
'Descending' => 'DESC'
],
],
'exclude_trailers' => [
'name' => 'Exclude trailers',
'type' => 'checkbox',
'required' => false,
'defaultValue' => false
],
],
'Category' => [
'lang' => [
'type' => 'list',
'name' => 'Language',
'values' => [
'Français' => 'fr',
'Deutsch' => 'de',
'English' => 'en',
'Español' => 'es',
'Polski' => 'pl',
'Italiano' => 'it'
],
],
'cat' => [
'type' => 'list',
'name' => 'Category',
'values' => [
'All videos' => null,
'News & society' => 'ACT',
'Series & fiction' => 'SER',
'Cinema' => 'CIN',
'Culture' => 'ARS',
'Culture pop' => 'CPO',
'Discovery' => 'DEC',
'History' => 'HIST',
'Science' => 'SCI',
'Other' => 'AUT'
]
],
],
'Collection' => [
'lang' => [
'type' => 'list',
'name' => 'Language',
'values' => [
'Français' => 'fr',
'Deutsch' => 'de',
'English' => 'en',
'Español' => 'es',
'Polski' => 'pl',
'Italiano' => 'it'
]
],
'col' => [
'name' => 'Collection id',
'required' => true,
'title' => 'ex. RC-014095 pour https://www.arte.tv/de/videos/RC-014095/blow-up/',
'exampleValue' => 'RC-014095'
]
]
];
public function collectData(){
switch($this->queriedContext) {
case 'Catégorie (Français)':
$category = $this->getInput('catfr');
$lang = 'fr';
break;
case 'Catégorie (Allemand)':
$category = $this->getInput('catde');
$lang = 'de';
break;
}
public function collectData()
{
switch ($this->queriedContext) {
case 'Category':
$category = $this->getInput('cat');
$collectionId = null;
break;
case 'Collection':
$collectionId = $this->getInput('col');
$category = null;
break;
}
$url = 'https://api.arte.tv/api/opa/v3/videos?sort=-lastModified&limit=10&language='
. $lang
. ($category != null ? '&category.code=' . $category : '');
$lang = $this->getInput('lang');
$sort_by = $this->getInput('sort_by');
$sort_direction = $this->getInput('sort_direction') == 'ASC' ? '' : '-';
$context = array(
'http' => array(
'header' => 'Authorization: Bearer '. self::API_TOKEN
)
);
$url = 'https://api.arte.tv/api/opa/v3/videos?limit=15&language='
. $lang
. ($sort_by != null ? '&sort=' . $sort_direction . $sort_by : '')
. ($category != null ? '&category.code=' . $category : '')
. ($collectionId != null ? '&collections.collectionId=' . $collectionId : '');
$input = getContents($url, false, stream_context_create($context)) or die('Could not request ARTE.');
$input_json = json_decode($input, true);
$header = [
'Authorization: Bearer ' . self::API_TOKEN
];
foreach($input_json['videos'] as $element) {
$input = getContents($url, $header);
$input_json = json_decode($input, true);
$item = array();
$item['uri'] = $element['url'];
$item['id'] = $element['id'];
foreach ($input_json['videos'] as $element) {
if ($this->getInput('exclude_trailers') && $element['platform'] == 'EXTRAIT') {
continue;
}
$item['timestamp'] = strtotime($element['videoRightsBegin']);
$item['title'] = $element['title'];
$durationSeconds = $element['durationSeconds'];
if(!empty($element['subtitle']))
$item['title'] = $element['title'] . ' | ' . $element['subtitle'];
$item = [];
$item['uri'] = $element['url'];
$item['id'] = $element['id'];
$item['duration'] = round((int)$element['durationSeconds'] / 60);
$item['content'] = $element['teaserText']
. '<br><br>'
. $item['duration']
. 'min<br><a href="'
. $item['uri']
. '"><img src="'
. $element['mainImage']['url']
. '" /></a>';
$item['timestamp'] = strtotime($element['videoRightsBegin']);
$item['title'] = $element['title'];
$this->items[] = $item;
}
}
if (!empty($element['subtitle'])) {
$item['title'] = $element['title'] . ' | ' . $element['subtitle'];
}
$durationMinutes = round((int)$durationSeconds / 60);
$item['content'] = $element['teaserText']
. '<br><br>'
. $durationMinutes
. 'min<br><a href="'
. $item['uri']
. '"><img src="'
. $element['mainImage']['url']
. '" /></a>';
$item['itunes'] = [
'duration' => $durationSeconds,
];
$this->items[] = $item;
}
}
}

View File

@@ -0,0 +1,80 @@
<?php
class AsahiShimbunAJWBridge extends BridgeAbstract
{
const NAME = 'Asahi Shimbun AJW';
const BASE_URI = 'http://www.asahi.com';
const URI = self::BASE_URI . '/ajw/';
const DESCRIPTION = 'Asahi Shimbun - Asia & Japan Watch';
const MAINTAINER = 'somini';
const PARAMETERS = [
[
'section' => [
'type' => 'list',
'name' => 'Section',
'values' => [
'Japan » Social Affairs' => 'japan/social',
'Japan » People' => 'japan/people',
'Japan » 3/11 Disaster' => 'japan/0311disaster',
'Japan » Sci & Tech' => 'japan/sci_tech',
'Politics' => 'politics',
'Business' => 'business',
'Culture » Style' => 'culture/style',
'Culture » Movies' => 'culture/movies',
'Culture » Manga & Anime' => 'culture/manga_anime',
'Asia » China' => 'asia_world/china',
'Asia » Korean Peninsula' => 'asia_world/korean_peninsula',
'Asia » Around Asia' => 'asia_world/around_asia',
'Asia » World' => 'asia_world/world',
'Opinion » Editorial' => 'opinion/editorial',
'Opinion » Vox Populi' => 'opinion/vox',
],
'defaultValue' => 'politics',
]
]
];
private function getSectionURI($section)
{
return $this->getURI() . $section . '/';
}
public function collectData()
{
$html = getSimpleHTMLDOM($this->getSectionURI($this->getInput('section')));
foreach ($html->find('#MainInner li a') as $element) {
if ($element->parent()->class == 'HeadlineTopImage-S') {
continue;
}
$item = [];
$item['uri'] = self::BASE_URI . $element->href;
$e_lead = $element->find('span.Lead', 0);
if ($e_lead) {
$item['content'] = $e_lead->innertext;
$e_lead->outertext = '';
} else {
$item['content'] = $element->innertext;
}
$e_date = $element->find('span.EnDate', 0);
if ($e_date) {
$item['timestamp'] = strtotime($e_date->innertext);
$e_date->outertext = '';
}
$e_video = $element->find('span.EnVideo', 0);
if ($e_video) {
$e_video->outertext = '';
$element->innertext = "VIDEO: $element->innertext";
}
$e_title = $element->find('.title', 0);
if ($e_title) {
$item['title'] = $e_title->innertext;
} else {
$item['title'] = $element->innertext;
}
$this->items[] = $item;
}
}
}

View File

@@ -1,74 +0,0 @@
<?php
class AskfmBridge extends BridgeAbstract {
const MAINTAINER = 'az5he6ch';
const NAME = 'Ask.fm Answers';
const URI = 'https://ask.fm/';
const CACHE_TIMEOUT = 300; //5 min
const DESCRIPTION = 'Returns answers from an Ask.fm user';
const PARAMETERS = array(
'Ask.fm username' => array(
'u' => array(
'name' => 'Username',
'required' => true
)
)
);
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('Requested username can\'t be found.');
foreach($html->find('div.streamItem-answer') as $element) {
$item = array();
$item['uri'] = self::URI . $element->find('a.streamItemsAge', 0)->href;
$question = trim($element->find('h1.streamItemContent-question', 0)->innertext);
$item['title'] = trim(
htmlspecialchars_decode($element->find('h1.streamItemContent-question', 0)->plaintext,
ENT_QUOTES
)
);
$answer = trim($element->find('p.streamItemContent-answer', 0)->innertext);
// Doesn't work, DOM parser doesn't seem to like data-hint, dunno why
#$item['update'] = $element->find('a.streamitemsage',0)->data-hint;
// This probably should be cleaned up, especially for YouTube embeds
$visual = $element->find('div.streamItemContent-visual', 0)->innertext;
//Fix tracking links, also doesn't work
foreach($element->find('a') as $link) {
if(strpos($link->href, 'l.ask.fm') !== false) {
// Too slow
#$link->href = str_replace('#_=_', '', get_headers($link->href, 1)['Location']);
$link->href = $link->plaintext;
}
}
$content = '<p>' . $question . '</p><p>' . $answer . '</p><p>' . $visual . '</p>';
// Fix relative links without breaking // scheme used by YouTube stuff
$content = preg_replace('#href="\/(?!\/)#', 'href="' . self::URI, $content);
$item['content'] = $content;
$this->items[] = $item;
}
}
public function getName(){
if(!is_null($this->getInput('u'))) {
return self::NAME . ' : ' . $this->getInput('u');
}
return parent::getName();
}
public function getURI(){
if(!is_null($this->getInput('u'))) {
return self::URI . urlencode($this->getInput('u')) . '/answers/more?page=0';
}
return parent::getURI();
}
}

View File

@@ -0,0 +1,280 @@
<?php
class AssociatedPressNewsBridge extends BridgeAbstract
{
const NAME = 'Associated Press News Bridge';
const URI = 'https://apnews.com/';
const DESCRIPTION = 'Returns newest articles by topic';
const MAINTAINER = 'VerifiedJoseph';
const PARAMETERS = [
'Standard Topics' => [
'topic' => [
'name' => 'Topic',
'type' => 'list',
'values' => [
'AP Top News' => 'apf-topnews',
'Sports' => 'apf-sports',
'Entertainment' => 'apf-entertainment',
'Oddities' => 'apf-oddities',
'Travel' => 'apf-Travel',
'Technology' => 'apf-technology',
'Lifestyle' => 'apf-lifestyle',
'Business' => 'apf-business',
'U.S. News' => 'apf-usnews',
'Health' => 'apf-Health',
'Science' => 'apf-science',
'World News' => 'apf-WorldNews',
'Politics' => 'apf-politics',
'Religion' => 'apf-religion',
'Photo Galleries' => 'PhotoGalleries',
'Fact Checks' => 'APFactCheck',
'Videos' => 'apf-videos',
],
'defaultValue' => 'apf-topnews',
],
],
'Custom Topic' => [
'topic' => [
'name' => 'Topic',
'type' => 'text',
'required' => true,
'exampleValue' => 'europe'
],
]
];
const CACHE_TIMEOUT = 900; // 15 mins
private $detectParamRegex = '/^https?:\/\/(?:www\.)?apnews\.com\/(?:[tag|hub]+\/)?([\w-]+)$/';
private $tagEndpoint = 'https://afs-prod.appspot.com/api/v2/feed/tag?tags=';
private $feedName = '';
public function detectParameters($url)
{
$params = [];
if (preg_match($this->detectParamRegex, $url, $matches) > 0) {
$params['topic'] = $matches[1];
$params['context'] = 'Custom Topic';
return $params;
}
return null;
}
public function collectData()
{
switch ($this->getInput('topic')) {
case 'Podcasts':
throwClientException('Podcasts topic feed is not supported');
break;
case 'PressReleases':
throwClientException('PressReleases topic feed is not supported');
break;
default:
$this->collectCardData();
}
}
public function getURI()
{
if (!is_null($this->getInput('topic'))) {
return self::URI . $this->getInput('topic');
}
return parent::getURI();
}
public function getName()
{
if (!empty($this->feedName)) {
return $this->feedName . ' - Associated Press';
}
return parent::getName();
}
private function getTagURI()
{
if (!is_null($this->getInput('topic'))) {
return $this->tagEndpoint . $this->getInput('topic');
}
return parent::getURI();
}
private function collectCardData()
{
$json = getContents($this->getTagURI());
$tagContents = json_decode($json, true);
if (empty($tagContents['tagObjs'])) {
throwClientException('Topic not found: ' . $this->getInput('topic'));
}
$this->feedName = $tagContents['tagObjs'][0]['name'];
foreach ($tagContents['cards'] as $card) {
$item = [];
// skip hub peeks & Notifications
if ($card['cardType'] == 'Hub Peek' || $card['cardType'] == 'Notification') {
continue;
}
$storyContent = $card['contents'][0];
switch ($storyContent['contentType']) {
case 'web': // Skip link only content
continue 2;
case 'video':
$html = $this->processVideo($storyContent);
$item['enclosures'][] = 'https://storage.googleapis.com/afs-prod/media/'
. $storyContent['media'][0]['id'] . '/800.jpeg';
break;
default:
if (empty($storyContent['storyHTML'])) { // Skip if no storyHTML
continue 2;
}
$html = defaultLinkTo($storyContent['storyHTML'], self::URI);
$html = str_get_html($html);
$this->processMediaPlaceholders($html, $storyContent['id']);
$this->processHubLinks($html, $storyContent);
$this->processIframes($html);
if (!is_null($storyContent['leadPhotoId'])) {
$leadPhotoUrl = sprintf('https://storage.googleapis.com/afs-prod/media/%s/800.jpeg', $storyContent['leadPhotoId']);
$leadPhotoImageTag = sprintf('<img src="%s">', $leadPhotoUrl);
// Move the image to the beginning of the content
$html = $leadPhotoImageTag . $html;
// Explicitly not adding it to the item's enclosures!
}
}
$item['title'] = $card['contents'][0]['headline'];
$item['uri'] = self::URI . $card['shortId'];
if ($card['contents'][0]['localLinkUrl']) {
$item['uri'] = $card['contents'][0]['localLinkUrl'];
}
$item['timestamp'] = $storyContent['published'];
if (is_null($storyContent['bylines']) === false) {
// Remove 'By' from the bylines
if (substr($storyContent['bylines'], 0, 2) == 'By') {
$item['author'] = ltrim($storyContent['bylines'], 'By ');
} else {
$item['author'] = $storyContent['bylines'];
}
}
$item['content'] = $html;
foreach ($storyContent['tagObjs'] as $tag) {
$item['categories'][] = $tag['name'];
}
$this->items[] = $item;
if (count($this->items) >= 15) {
break;
}
}
}
private function processMediaPlaceholders($html, $id)
{
if ($html->find('div.media-placeholder', 0)) {
// Fetch page content
$json = getContents('https://afs-prod.appspot.com/api/v2/content/' . $id);
$storyContent = json_decode($json, true);
foreach ($html->find('div.media-placeholder') as $div) {
$key = array_search($div->id, $storyContent['mediumIds']);
if (!isset($storyContent['media'][$key])) {
continue;
}
$media = $storyContent['media'][$key];
if ($media['type'] === 'Photo') {
$mediaUrl = $media['gcsBaseUrl'] . $media['imageRenderedSizes'][0] . $media['imageFileExtension'];
$mediaCaption = $media['caption'];
$div->outertext = <<<EOD
<figure><img loading="lazy" src="{$mediaUrl}"/><figcaption>{$mediaCaption}</figcaption></figure>
EOD;
}
if ($media['type'] === 'YouTube') {
$div->outertext = <<<EOD
<iframe src="https://www.youtube.com/embed/{$media['externalId']}" width="560" height="315">
</iframe>
EOD;
}
}
}
}
/*
Create full coverage links (HubLinks)
*/
private function processHubLinks($html, $storyContent)
{
if (!empty($storyContent['richEmbeds'])) {
foreach ($storyContent['richEmbeds'] as $embed) {
if ($embed['type'] === 'Hub Link') {
$url = self::URI . $embed['tag']['id'];
$div = $html->find('div[id=' . $embed['id'] . ']', 0);
if ($div) {
$div->outertext = <<<EOD
<p><a href="{$url}">{$embed['calloutText']} {$embed['displayName']}</a></p>
EOD;
}
}
}
}
}
private function processVideo($storyContent)
{
$video = $storyContent['media'][0];
if ($video['type'] === 'YouTube') {
$url = 'https://www.youtube.com/embed/' . $video['externalId'];
$html = <<<EOD
<iframe width="560" height="315" src="{$url}" frameborder="0" allowfullscreen></iframe>
EOD;
} else {
$html = <<<EOD
<video controls poster="https://storage.googleapis.com/afs-prod/media/{$video['id']}/800.jpeg" preload="none">
<source src="{$video['gcsBaseUrl']} {$video['videoRenderedSizes'][0]} {$video['videoFileExtension']}" type="video/mp4">
</video>
EOD;
}
return $html;
}
// Remove datawrapper.dwcdn.net iframes and related javaScript
private function processIframes($html)
{
foreach ($html->find('iframe') as $index => $iframe) {
if (preg_match('/datawrapper\.dwcdn\.net/', $iframe->src)) {
$iframe->outertext = '';
if ($html->find('script', $index)) {
$html->find('script', $index)->outertext = '';
}
}
}
}
}

View File

@@ -0,0 +1,53 @@
<?php
class AstrophysicsDataSystemBridge extends BridgeAbstract
{
const NAME = 'SAO/NASA Astrophysics Data System';
const DESCRIPTION = 'Returns the latest publications from a query';
const URI = 'https://ui.adsabs.harvard.edu';
const PARAMETERS = [
'Publications' => [
'query' => [
'name' => 'query',
'title' => 'Same format as the search bar on the website',
'exampleValue' => 'author:"huchra, john"',
'required' => true
]
]];
private $feedTitle;
public function getName()
{
if ($this->queriedContext === 'Publications') {
return $this->feedTitle;
}
return parent::getName();
}
public function getURI()
{
if ($this->queriedContext === 'Publications') {
return self::URI . '/search/?q=' . urlencode($this->getInput('query'));
}
return parent::getURI();
}
public function collectData()
{
$headers = [
'Cookie: core=always;'
];
$html = str_get_html(defaultLinkTo(getContents($this->getURI(), $headers), self::URI));
$this->feedTitle = html_entity_decode($html->find('title', 0)->plaintext);
foreach ($html->find('div.row > ul > li') as $pub) {
$item = [];
$item['title'] = $pub->find('h3.s-results-title', 0)->plaintext;
$item['content'] = $pub->find('div.s-results-links', 0);
$item['uri'] = $pub->find('a.abs-redirect-link', 0)->href;
$item['author'] = rtrim($pub->find('li.article-author', 0)->plaintext, ' ;');
$item['timestamp'] = $pub->find('div[aria-label="date published"]', 0)->plaintext;
$this->items[] = $item;
}
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,60 @@
<?php
class AtmoOccitanieBridge extends BridgeAbstract
{
const NAME = 'Atmo Occitanie';
const URI = 'https://www.atmo-occitanie.org/';
const DESCRIPTION = 'Fetches the latest air polution of cities in Occitanie from Atmo';
const MAINTAINER = 'floviolleau';
const PARAMETERS = [[
'city' => [
'name' => 'Ville',
'required' => true,
'exampleValue' => 'cahors'
]
]];
const CACHE_TIMEOUT = 7200;
public function collectData()
{
$uri = self::URI . $this->getInput('city');
$html = getSimpleHTMLDOM($uri);
$generalMessage = $html->find('.landing-ville .city-banner .iqa-avertissement', 0)->innertext;
$recommendationsDom = $html->find('.landing-ville .recommandations', 0);
$recommendationsItemDom = $recommendationsDom->find('.recommandation-item .label');
$recommendationsMessage = '';
$i = 0;
$len = count($recommendationsItemDom);
foreach ($recommendationsItemDom as $key => $value) {
if ($i == 0) {
$recommendationsMessage .= trim($value->innertext) . '.';
} else {
$recommendationsMessage .= ' ' . trim($value->innertext) . '.';
}
$i++;
}
$lastRecommendationsDom = $recommendationsDom->find('.col-md-6', -1);
$informationHeaderMessage = $lastRecommendationsDom->find('.heading', 0)->innertext;
$indice = $lastRecommendationsDom->find('.current-indice .indice div', 0)->innertext;
$informationDescriptionMessage = $lastRecommendationsDom->find('.current-indice .description p', 0)->innertext;
$message = "$generalMessage L'indice est de $indice/10. $informationDescriptionMessage. $recommendationsMessage";
$city = $this->getInput('city');
$item['uri'] = $uri;
$today = date('d/m/Y');
$item['title'] = "Bulletin de l'air du $today pour la ville : $city.";
//$item['title'] .= ' Retrouvez plus d\'informations en allant sur atmo-occitanie.org #QualiteAir. ' . $message;
$item['title'] .= ' #QualiteAir. ' . $message;
$item['author'] = 'floviolleau';
$item['content'] = $message;
$item['uid'] = hash('sha256', $item['title']);
$this->items[] = $item;
}
}

344
bridges/AuctionetBridge.php Normal file
View File

@@ -0,0 +1,344 @@
<?php
class AuctionetBridge extends BridgeAbstract
{
const NAME = 'Auctionet';
const URI = 'https://www.auctionet.com';
const DESCRIPTION = 'Fetches info about auction objects from Auctionet (an auction platform for many European auction houses)';
const MAINTAINER = 'Qluxzz';
const PARAMETERS = [[
'category' => [
'name' => 'Category',
'type' => 'list',
'values' => [
'All categories' => '',
'Art' => [
'All' => '25-art',
'Drawings' => '119-drawings',
'Engravings & Prints' => '27-engravings-prints',
'Other' => '30-other',
'Paintings' => '28-paintings',
'Photography' => '26-photography',
'Sculptures & Bronzes' => '29-sculptures-bronzes',
],
'Asiatica' => [
'All' => '117-asiatica',
],
'Books, Maps & Manuscripts' => [
'All' => '50-books-maps-manuscripts',
'Autographs & Manuscripts' => '206-autographs-manuscripts',
'Books' => '204-books',
'Maps' => '205-maps',
'Other' => '207-other',
],
'Carpets & Textiles' => [
'All' => '35-carpets-textiles',
'Carpets' => '36-carpets',
'Textiles' => '37-textiles',
],
'Ceramics & Porcelain' => [
'All' => '9-ceramics-porcelain',
'European' => '10-european',
'Oriental' => '11-oriental',
'Rest of the world' => '12-rest-of-the-world',
'Tableware' => '210-tableware',
],
'Clocks & Watches' => [
'All' => '31-clocks-watches',
'Carriage & Miniature Clocks' => '258-carriage-miniature-clocks',
'Longcase clocks' => '32-longcase-clocks',
'Mantel clocks' => '33-mantel-clocks',
'Other clocks' => '34-other-clocks',
'Pocket & Stop Watches' => '110-pocket-stop-watches',
'Wall Clocks' => '127-wall-clocks',
'Wristwatches' => '15-wristwatches',
],
'Coins, Medals & Stamps' => [
'All' => '46-coins-medals-stamps',
'Coins' => '128-coins',
'Orders & Medals' => '135-orders-medals',
'Other' => '131-other',
'Stamps' => '136-stamps',
],
'Folk art' => [
'All' => '58-folk-art',
'Bowls & Boxes' => '121-bowls-boxes',
'Furniture' => '122-furniture',
'Other' => '123-other',
'Tools & Gears' => '120-tools-gears',
],
'Furniture' => [
'All' => '16-furniture',
'Armchairs & Chairs' => '18-armchairs-chairs',
'Chests of drawers' => '24-chests-of-drawers',
'Cupboards, Cabinets & Shelves' => '23-cupboards-cabinets-shelves',
'Dining room furniture' => '22-dining-room-furniture',
'Garden' => '21-garden',
'Other' => '17-other',
'Sofas & seatings' => '20-sofas-seatings',
'Tables' => '19-tables',
],
'Glass' => [
'All' => '6-glass',
'Art glass' => '208-art-glass',
'Other' => '8-other',
'Tableware' => '7-tableware',
'Utility glass' => '209-utility-glass',
],
'Jewellery & Gemstones' => [
'All' => '13-jewellery-gemstones',
'Alliance rings' => '113-alliance-rings',
'Bracelets' => '106-bracelets',
'Brooches & Pendants' => '107-brooches-pendants',
'Costume Jewellery' => '259-costume-jewellery',
'Cufflinks & Tie Pins' => '111-cufflinks-tie-pins',
'Ear studs' => '116-ear-studs',
'Earrings' => '115-earrings',
'Gemstones' => '48-gemstones',
'Jewellery' => '14-jewellery',
'Jewellery Suites' => '109-jewellery-suites',
'Necklace' => '104-necklace',
'Other' => '118-other',
'Rings' => '112-rings',
'Signet rings' => '105-signet-rings',
'Solitaire rings' => '114-solitaire-rings',
],
'Licence weapons' => [
'All' => '59-licence-weapons',
'Combi/Combo' => '63-combi-combo',
'Double express rifles' => '60-double-express-rifles',
'Rifles' => '61-rifles',
'Shotguns' => '62-shotguns',
],
'Lighting & Lamps' => [
'All' => '1-lighting-lamps',
'Candlesticks' => '4-candlesticks',
'Ceiling lights' => '3-ceiling-lights',
'Chandeliers' => '203-chandeliers',
'Floor lights' => '2-floor-lights',
'Other lighting' => '5-other-lighting',
'Table Lamps' => '125-table-lamps',
'Wall Lights' => '124-wall-lights',
],
'Mirrors' => [
'All' => '42-mirrors',
],
'Miscellaneous' => [
'All' => '43-miscellaneous',
'Fishing equipment' => '54-fishing-equipment',
'Miscellaneous' => '47-miscellaneous',
'Modern Tools' => '133-modern-tools',
'Modern consumer electronics' => '52-modern-consumer-electronics',
'Musical instruments' => '51-musical-instruments',
'Technica & Nautica' => '45-technica-nautica',
],
'Photo, Cameras & Lenses' => [
'All' => '57-photo-cameras-lenses',
'Cameras & accessories' => '71-cameras-accessories',
'Optics' => '66-optics',
'Other' => '72-other',
],
'Silver & Metals' => [
'All' => '38-silver-metals',
'Other metals' => '40-other-metals',
'Pewter, Brass & Copper' => '41-pewter-brass-copper',
'Silver' => '39-silver',
'Silver plated' => '213-silver-plated',
],
'Toys' => [
'All' => '44-toys',
'Comics' => '211-comics',
'Toys' => '212-toys',
],
'Tribal art' => [
'All' => '134-tribal-art',
],
'Vehicles, Boats & Parts' => [
'All' => '249-vehicles-boats-parts',
'Automobilia & Transport' => '255-automobilia-transport',
'Bicycles' => '132-bicycles',
'Boats & Accessories' => '250-boats-accessories',
'Car parts' => '253-car-parts',
'Cars' => '215-cars',
'Moped parts' => '254-moped-parts',
'Mopeds' => '216-mopeds',
'Motorcycle parts' => '252-motorcycle-parts',
'Motorcycles' => '251-motorcycles',
'Other' => '256-other',
],
'Vintage & Designer Fashion' => [
'All' => '49-vintage-designer-fashion',
],
'Weapons & Militaria' => [
'All' => '137-weapons-militaria',
'Airguns' => '257-airguns',
'Armour & Uniform' => '138-armour-uniform',
'Edged weapons' => '130-edged-weapons',
'Guns & Rifles' => '129-guns-rifles',
'Other' => '214-other',
],
'Wine, Port & Spirits' => [
'All' => '170-wine-port-spirits',
],
]
],
'sort_order' => [
'name' => 'Sort order',
'type' => 'list',
'values' => [
'Most bids' => 'bids_count_desc',
'Lowest bid' => 'bid_asc',
'Highest bid' => 'bid_desc',
'Last bid on' => 'bid_on',
'Ending soonest' => 'end_asc_active',
'Lowest estimate' => 'estimate_asc',
'Highest estimate' => 'estimate_desc',
'Recently added' => 'recent'
],
],
'country' => [
'name' => 'Country',
'type' => 'list',
'values' => [
'All' => '',
'Denmark' => 'DK',
'Finland' => 'FI',
'Germany' => 'DE',
'Spain' => 'ES',
'Sweden' => 'SE',
'United Kingdom' => 'GB'
]
],
'language' => [
'name' => 'Language',
'type' => 'list',
'values' => [
'English' => 'en',
'Español' => 'es',
'Deutsch' => 'de',
'Svenska' => 'sv',
'Dansk' => 'da',
'Suomi' => 'fi',
],
],
]];
const CACHE_TIMEOUT = 3600; // 1 hour
private $title;
public function collectData()
{
// Each page contains 48 auctions
// So we fetch 10 pages so we decrease the likelihood
// of missing auctions between feed refreshes
// Fetch first page and use that to get title
{
$url = $this->getUrl(1);
$data = getContents($url);
$title = $this->getDocumentTitle($data);
$this->items = array_merge($this->items, $this->parsePageData($data));
}
// Fetch remaining pages
for ($page = 2; $page <= 10; $page++) {
$url = $this->getUrl($page);
$data = getContents($url);
$this->items = array_merge($this->items, $this->parsePageData($data));
}
}
public function getName()
{
return $this->title ?: parent::getName();
}
/* HELPERS */
private function getUrl($page)
{
$category = $this->getInput('category');
$language = $this->getInput('language');
$sort_order = $this->getInput('sort_order');
$country = $this->getInput('country');
$url = self::URI . '/' . $language . '/search';
if ($category) {
$url = $url . '/' . $category;
}
$query = [];
$query['page'] = $page;
if ($sort_order) {
$query['order'] = $sort_order;
}
if ($country) {
$query['country_code'] = $country;
}
if (count($query) > 0) {
$url = $url . '?' . http_build_query($query);
}
return $url;
}
private function getDocumentTitle($data)
{
$title_elem = '<title>';
$title_elem_length = strlen($title_elem);
$title_start = strpos($data, $title_elem);
$title_end = strpos($data, '</title>', $title_start);
$title_length = $title_end - $title_start + strlen($title_elem);
$title = substr($data, $title_start + strlen($title_elem), $title_length);
return $title;
}
/**
* The auction items data is included in the HTML document
* as a HTML entities encoded JSON structure
* which is used to hydrate the React component for the list of auctions
*/
private function parsePageData($data)
{
$key = 'data-react-props="';
$keyLength = strlen($key);
$start = strpos($data, $key);
$end = strpos($data, '"', $start + strlen($key));
$length = $end - ($start + $keyLength);
$jsonString = substr($data, $start + $keyLength, $length);
$jsonData = json_decode(htmlspecialchars_decode($jsonString), false);
$items = [];
foreach ($jsonData->{'items'} as $item) {
$title = $item->{'longTitle'};
$relative_url = $item->{'url'};
$images = $item->{'imageUrls'};
$id = $item->{'auctionId'};
$items[] = [
'title' => $title,
'uri' => self::URI . $relative_url,
'uid' => $id,
'content' => count($images) > 0 ? "<img src='$images[0]'/><br/>$title" : $title,
'enclosures' => array_slice($images, 1),
];
}
return $items;
}
}

168
bridges/AutoJMBridge.php Normal file
View File

@@ -0,0 +1,168 @@
<?php
class AutoJMBridge extends BridgeAbstract
{
const NAME = 'AutoJM';
const URI = 'https://www.autojm.fr/';
const DESCRIPTION = 'Suivre les offres de véhicules proposés par AutoJM en fonction des critères de filtrages';
const MAINTAINER = 'sysadminstory';
const PARAMETERS = [
'Afficher les offres de véhicules disponible sur la recheche AutoJM' => [
'url' => [
'name' => 'URL de la page de recherche',
'type' => 'text',
'required' => true,
'title' => 'URL d\'une recherche avec filtre de véhicules sans le http://www.autojm.fr/',
'exampleValue' => 'recherche?brands[]=PEUGEOT&ranges[]=PEUGEOT 308'
],
]
];
const CACHE_TIMEOUT = 3600;
const TEST_DETECT_PARAMETERS = [
'https://www.autojm.fr/recherche?brands%5B%5D=PEUGEOT&ranges%5B%5D=PEUGEOT%20308'
=> ['url' => 'recherche?brands%5B%5D=PEUGEOT&ranges%5B%5D=PEUGEOT%20308',
'context' => 'Afficher les offres de véhicules disponible sur la recheche AutoJM'
]
];
public function getIcon()
{
return self::URI . 'favicon.ico';
}
public function getName()
{
switch ($this->queriedContext) {
case 'Afficher les offres de véhicules disponible sur la recheche AutoJM':
return 'AutoJM | Recherche de véhicules';
break;
default:
return parent::getName();
}
}
public function getURI()
{
switch ($this->queriedContext) {
case 'Afficher les offres de véhicules disponible sur la recheche AutoJM':
return self::URI . $this->getInput('url');
break;
default:
return self::URI;
}
}
public function collectData()
{
// Get the number of result for this search
$search_url = self::URI . $this->getInput('url') . '&open=energy&onlyFilters=false';
// Set the header 'X-Requested-With' like the website does it
$header = [
'X-Requested-With: XMLHttpRequest'
];
// Get the JSON content of the form
$json = getContents($search_url, $header);
// Extract the HTML content from the JSON result
$data = json_decode($json);
$nb_results = $data->nbResults;
$total_pages = ceil($nb_results / 14);
// Limit the number of page to analyse to 10
for ($page = 1; $page <= $total_pages && $page <= 10; $page++) {
// Get the result the next page
$html = $this->getResults($page);
// Go through every car of the search
$list = $html->find('div[class*=card-car card-car--listing]');
foreach ($list as $car) {
// Get the info about the car offer
$image = $car->find('div[class=card-car__header__img]', 0)->find('img', 0)->src;
// Decode HTML attribute JSON data
$car_data = json_decode(html_entity_decode($car->{'data-layer'}));
$car_model = $car_data->title;
$availability = $car->find('div[class*=card-car__modalites]', 0)->find('div[class=col]', 0)->plaintext;
$warranty = $car->find('div[data-type=WarrantyCard]', 0)->plaintext;
$discount_html = $car->find('div[class=subtext vehicle_reference_element]', 0);
// Check if there is any discount info displayed
if ($discount_html != null) {
$reference_price_value = $discount_html->find('span[data-cfg=vehicle__reference_price]', 0)->plaintext;
$discount_percent_value = $discount_html->find('span[data-cfg=vehicle__discount_percent]', 0)->plaintext;
$reference_price = '<li>Prix de référence : <s>' . $reference_price_value . '</s></li>';
$discount_percent = '<li>Réduction : ' . $discount_percent_value . ' %</li>';
} else {
$reference_price = '';
$discount_percent = '';
}
$price = $car_data->price;
$kilometer = $car->find('span[data-cfg=vehicle__kilometer]', 0)->plaintext;
$energy = $car->find('span[data-cfg=vehicle__energy__label]', 0)->plaintext;
$power = $car->find('span[data-cfg=vehicle__tax_horse_power]', 0)->plaintext;
$seats = $car->find('span[data-cfg=vehicle__seats]', 0)->plaintext;
$doors = $car->find('span[data-cfg=vehicle__door__label]', 0)->plaintext;
$transmission = $car->find('span[data-cfg=vehicle__transmission]', 0)->plaintext;
$loa_html = $car->find('span[data-cfg=vehicle__loa]', 0);
// Check if any LOA price is displayed
if ($loa_html != null) {
$loa_value = $car->find('span[data-cfg=vehicle__loa]', 0)->plaintext;
$loa = '<li>LOA : à partir de ' . $loa_value . ' / mois </li>';
} else {
$loa = '';
}
// Construct the new item
$item = [];
$item['title'] = $car_model;
$item['content'] = '<p><img style="vertical-align:middle ; padding: 10px" src="' . $image . '" />'
. $car_model . '</p>';
$item['content'] .= '<ul><li>Disponibilité : ' . $availability . '</li>';
$item['content'] .= '<li>Prix : ' . $price . ' €</li>';
$item['content'] .= $reference_price;
$item['content'] .= $loa;
$item['content'] .= $discount_percent;
$item['content'] .= '<li>Garantie : ' . $warranty . '</li>';
$item['content'] .= '<li>Kilométrage : ' . $kilometer . ' km</li>';
$item['content'] .= '<li>Energie : ' . $energy . '</li>';
$item['content'] .= '<li>Puissance: ' . $power . ' CV Fiscaux</li>';
$item['content'] .= '<li>Nombre de Places : ' . $seats . ' place(s)</li>';
$item['content'] .= '<li>Nombre de portes : ' . $doors . '</li>';
$item['content'] .= '<li>Boite de vitesse : ' . $transmission . '</li></ul>';
$item['uri'] = $car_data->{'uri'};
$item['uid'] = hash('md5', $item['content']);
$this->items[] = $item;
}
}
}
private function getResults(int $page)
{
$user_input = $this->getInput('url');
$search_data = preg_replace('#(recherche|recherche/[0-9]{1,10})\?#', 'recherche/' . $page . '?', $user_input);
$search_url = self::URI . $search_data . '&open=energy&onlyFilters=false';
// Get the HTML content of the page
$html = getSimpleHTMLDOMCached($search_url);
return $html;
}
public function detectParameters($url)
{
$params = [];
$regex = '/^(https?:\/\/)?(www\.|)autojm.fr\/(recherche\?.*|recherche\/[0-9]{1,10}\?.*)$/m';
if (preg_match($regex, $url, $matches) > 0) {
$url = preg_replace('#(recherche|recherche/[0-9]{1,10})#', 'recherche', $matches[3]);
$params['url'] = $url;
$params['context'] = 'Afficher les offres de véhicules disponible sur la recheche AutoJM';
return $params;
}
}
}

View File

@@ -0,0 +1,59 @@
<?php
class AwwwardsBridge extends BridgeAbstract
{
const NAME = 'Awwwards';
const URI = 'https://www.awwwards.com/';
const DESCRIPTION = 'Fetches the latest ten sites of the day from Awwwards';
const MAINTAINER = 'Paroleen';
const CACHE_TIMEOUT = 3600;
const SITESURI = 'https://www.awwwards.com/websites/sites_of_the_day/';
const SITEURI = 'https://www.awwwards.com/sites/';
const ASSETSURI = 'https://assets.awwwards.com/awards/media/cache/thumb_417_299/';
private $sites = [];
public function collectData()
{
$this->fetchSites();
foreach ($this->sites as $site) {
$item = [];
$item['title'] = $site['title'];
$item['timestamp'] = $site['createdAt'];
$item['categories'] = $site['tags'];
$item['content'] = '<img src="'
. self::ASSETSURI
. $site['images']['thumbnail']
. '">';
$item['uri'] = self::SITEURI . $site['slug'];
$this->items[] = $item;
if (count($this->items) >= 10) {
break;
}
}
}
public function getIcon()
{
return 'https://www.awwwards.com/favicon.ico';
}
private function fetchSites()
{
$sites = getSimpleHTMLDOM(self::SITESURI);
foreach ($sites->find('.grid-sites li') as $li) {
$encodedJson = $li->attr['data-collectable-model-value'] ?? null;
if (!$encodedJson) {
continue;
}
$json = html_entity_decode($encodedJson, ENT_QUOTES, 'utf-8');
$site = Json::decode($json);
$this->sites[] = $site;
}
}
}

269
bridges/BAEBridge.php Normal file
View File

@@ -0,0 +1,269 @@
<?php
class BAEBridge extends BridgeAbstract
{
const MAINTAINER = 'couraudt';
const NAME = 'Bourse Aux Equipiers Bridge';
const URI = 'https://www.bourse-aux-equipiers.com';
const DESCRIPTION = 'Returns the newest sailing offers.';
const PARAMETERS = [
[
'keyword' => [
'name' => 'Filtrer par mots clés',
'title' => 'Entrez le mot clé à filtrer ici'
],
'type' => [
'name' => 'Type de recherche',
'title' => 'Afficher seuleument un certain type d\'annonce',
'type' => 'list',
'values' => [
'Toutes les annonces' => false,
'Les embarquements' => 'boat',
'Les skippers' => 'skipper',
'Les équipiers' => 'crew'
]
]
]
];
public function collectData()
{
$url = $this->getURI();
$html = getSimpleHTMLDOM($url);
$annonces = $html->find('main article');
foreach ($annonces as $annonce) {
$detail = $annonce->find('footer a', 0);
$htmlDetail = getSimpleHTMLDOMCached(parent::getURI() . $detail->href);
if (!$htmlDetail) {
continue;
}
$item = [];
$item['title'] = $annonce->find('header h2', 0)->plaintext;
$item['uri'] = parent::getURI() . $detail->href;
$content = $htmlDetail->find('article p', 0)->innertext;
if (!empty($this->getInput('keyword'))) {
$keyword = $this->removeAccents(strtolower($this->getInput('keyword')));
$cleanTitle = $this->removeAccents(strtolower($item['title']));
if (strpos($cleanTitle, $keyword) === false) {
$cleanContent = $this->removeAccents(strtolower($content));
if (strpos($cleanContent, $keyword) === false) {
continue;
}
}
}
$content .= '<hr>';
$content .= $htmlDetail->find('section', 0)->innertext;
$item['content'] = defaultLinkTo($content, parent::getURI());
$image = $htmlDetail->find('#zoom', 0);
if ($image) {
$item['enclosures'] = [parent::getURI() . $image->getAttribute('src')];
}
$this->items[] = $item;
}
}
public function getURI()
{
$uri = parent::getURI();
if (!empty($this->getInput('type'))) {
if ($this->getInput('type') == 'boat') {
$uri .= '/embarquements.html';
} elseif ($this->getInput('type') == 'skipper') {
$uri .= '/skippers.html';
} else {
$uri .= '/equipiers.html';
}
}
return $uri;
}
private function removeAccents($string)
{
$chars = [
// Decompositions for Latin-1 Supplement
'ª' => 'a', 'º' => 'o',
'À' => 'A', 'Á' => 'A',
'Â' => 'A', 'Ã' => 'A',
'Ä' => 'A', 'Å' => 'A',
'Æ' => 'AE', 'Ç' => 'C',
'È' => 'E', 'É' => 'E',
'Ê' => 'E', 'Ë' => 'E',
'Ì' => 'I', 'Í' => 'I',
'Î' => 'I', 'Ï' => 'I',
'Ð' => 'D', 'Ñ' => 'N',
'Ò' => 'O', 'Ó' => 'O',
'Ô' => 'O', 'Õ' => 'O',
'Ö' => 'O', 'Ù' => 'U',
'Ú' => 'U', 'Û' => 'U',
'Ü' => 'U', 'Ý' => 'Y',
'Þ' => 'TH', 'ß' => 's',
'à' => 'a', 'á' => 'a',
'â' => 'a', 'ã' => 'a',
'ä' => 'a', 'å' => 'a',
'æ' => 'ae', 'ç' => 'c',
'è' => 'e', 'é' => 'e',
'ê' => 'e', 'ë' => 'e',
'ì' => 'i', 'í' => 'i',
'î' => 'i', 'ï' => 'i',
'ð' => 'd', 'ñ' => 'n',
'ò' => 'o', 'ó' => 'o',
'ô' => 'o', 'õ' => 'o',
'ö' => 'o', 'ø' => 'o',
'ù' => 'u', 'ú' => 'u',
'û' => 'u', 'ü' => 'u',
'ý' => 'y', 'þ' => 'th',
'ÿ' => 'y', 'Ø' => 'O',
// Decompositions for Latin Extended-A
'Ā' => 'A', 'ā' => 'a',
'Ă' => 'A', 'ă' => 'a',
'Ą' => 'A', 'ą' => 'a',
'Ć' => 'C', 'ć' => 'c',
'Ĉ' => 'C', 'ĉ' => 'c',
'Ċ' => 'C', 'ċ' => 'c',
'Č' => 'C', 'č' => 'c',
'Ď' => 'D', 'ď' => 'd',
'Đ' => 'D', 'đ' => 'd',
'Ē' => 'E', 'ē' => 'e',
'Ĕ' => 'E', 'ĕ' => 'e',
'Ė' => 'E', 'ė' => 'e',
'Ę' => 'E', 'ę' => 'e',
'Ě' => 'E', 'ě' => 'e',
'Ĝ' => 'G', 'ĝ' => 'g',
'Ğ' => 'G', 'ğ' => 'g',
'Ġ' => 'G', 'ġ' => 'g',
'Ģ' => 'G', 'ģ' => 'g',
'Ĥ' => 'H', 'ĥ' => 'h',
'Ħ' => 'H', 'ħ' => 'h',
'Ĩ' => 'I', 'ĩ' => 'i',
'Ī' => 'I', 'ī' => 'i',
'Ĭ' => 'I', 'ĭ' => 'i',
'Į' => 'I', 'į' => 'i',
'İ' => 'I', 'ı' => 'i',
'IJ' => 'IJ', 'ij' => 'ij',
'Ĵ' => 'J', 'ĵ' => 'j',
'Ķ' => 'K', 'ķ' => 'k',
'ĸ' => 'k', 'Ĺ' => 'L',
'ĺ' => 'l', 'Ļ' => 'L',
'ļ' => 'l', 'Ľ' => 'L',
'ľ' => 'l', 'Ŀ' => 'L',
'ŀ' => 'l', 'Ł' => 'L',
'ł' => 'l', 'Ń' => 'N',
'ń' => 'n', 'Ņ' => 'N',
'ņ' => 'n', 'Ň' => 'N',
'ň' => 'n', 'ʼn' => 'n',
'Ŋ' => 'N', 'ŋ' => 'n',
'Ō' => 'O', 'ō' => 'o',
'Ŏ' => 'O', 'ŏ' => 'o',
'Ő' => 'O', 'ő' => 'o',
'Œ' => 'OE', 'œ' => 'oe',
'Ŕ' => 'R', 'ŕ' => 'r',
'Ŗ' => 'R', 'ŗ' => 'r',
'Ř' => 'R', 'ř' => 'r',
'Ś' => 'S', 'ś' => 's',
'Ŝ' => 'S', 'ŝ' => 's',
'Ş' => 'S', 'ş' => 's',
'Š' => 'S', 'š' => 's',
'Ţ' => 'T', 'ţ' => 't',
'Ť' => 'T', 'ť' => 't',
'Ŧ' => 'T', 'ŧ' => 't',
'Ũ' => 'U', 'ũ' => 'u',
'Ū' => 'U', 'ū' => 'u',
'Ŭ' => 'U', 'ŭ' => 'u',
'Ů' => 'U', 'ů' => 'u',
'Ű' => 'U', 'ű' => 'u',
'Ų' => 'U', 'ų' => 'u',
'Ŵ' => 'W', 'ŵ' => 'w',
'Ŷ' => 'Y', 'ŷ' => 'y',
'Ÿ' => 'Y', 'Ź' => 'Z',
'ź' => 'z', 'Ż' => 'Z',
'ż' => 'z', 'Ž' => 'Z',
'ž' => 'z', 'ſ' => 's',
// Decompositions for Latin Extended-B
'Ș' => 'S', 'ș' => 's',
'Ț' => 'T', 'ț' => 't',
// Euro Sign
'€' => 'E',
// GBP (Pound) Sign
'£' => '',
// Vowels with diacritic (Vietnamese)
// unmarked
'Ơ' => 'O', 'ơ' => 'o',
'Ư' => 'U', 'ư' => 'u',
// grave accent
'Ầ' => 'A', 'ầ' => 'a',
'Ằ' => 'A', 'ằ' => 'a',
'Ề' => 'E', 'ề' => 'e',
'Ồ' => 'O', 'ồ' => 'o',
'Ờ' => 'O', 'ờ' => 'o',
'Ừ' => 'U', 'ừ' => 'u',
'Ỳ' => 'Y', 'ỳ' => 'y',
// hook
'Ả' => 'A', 'ả' => 'a',
'Ẩ' => 'A', 'ẩ' => 'a',
'Ẳ' => 'A', 'ẳ' => 'a',
'Ẻ' => 'E', 'ẻ' => 'e',
'Ể' => 'E', 'ể' => 'e',
'Ỉ' => 'I', 'ỉ' => 'i',
'Ỏ' => 'O', 'ỏ' => 'o',
'Ổ' => 'O', 'ổ' => 'o',
'Ở' => 'O', 'ở' => 'o',
'Ủ' => 'U', 'ủ' => 'u',
'Ử' => 'U', 'ử' => 'u',
'Ỷ' => 'Y', 'ỷ' => 'y',
// tilde
'Ẫ' => 'A', 'ẫ' => 'a',
'Ẵ' => 'A', 'ẵ' => 'a',
'Ẽ' => 'E', 'ẽ' => 'e',
'Ễ' => 'E', 'ễ' => 'e',
'Ỗ' => 'O', 'ỗ' => 'o',
'Ỡ' => 'O', 'ỡ' => 'o',
'Ữ' => 'U', 'ữ' => 'u',
'Ỹ' => 'Y', 'ỹ' => 'y',
// acute accent
'Ấ' => 'A', 'ấ' => 'a',
'Ắ' => 'A', 'ắ' => 'a',
'Ế' => 'E', 'ế' => 'e',
'Ố' => 'O', 'ố' => 'o',
'Ớ' => 'O', 'ớ' => 'o',
'Ứ' => 'U', 'ứ' => 'u',
// dot below
'Ạ' => 'A', 'ạ' => 'a',
'Ậ' => 'A', 'ậ' => 'a',
'Ặ' => 'A', 'ặ' => 'a',
'Ẹ' => 'E', 'ẹ' => 'e',
'Ệ' => 'E', 'ệ' => 'e',
'Ị' => 'I', 'ị' => 'i',
'Ọ' => 'O', 'ọ' => 'o',
'Ộ' => 'O', 'ộ' => 'o',
'Ợ' => 'O', 'ợ' => 'o',
'Ụ' => 'U', 'ụ' => 'u',
'Ự' => 'U', 'ự' => 'u',
'Ỵ' => 'Y', 'ỵ' => 'y',
// Vowels with diacritic (Chinese, Hanyu Pinyin)
'ɑ' => 'a',
// macron
'Ǖ' => 'U', 'ǖ' => 'u',
// acute accent
'Ǘ' => 'U', 'ǘ' => 'u',
// caron
'Ǎ' => 'A', 'ǎ' => 'a',
'Ǐ' => 'I', 'ǐ' => 'i',
'Ǒ' => 'O', 'ǒ' => 'o',
'Ǔ' => 'U', 'ǔ' => 'u',
'Ǚ' => 'U', 'ǚ' => 'u',
// grave accent
'Ǜ' => 'U', 'ǜ' => 'u',
];
$string = strtr($string, $chars);
return $string;
}
}

View File

@@ -0,0 +1,254 @@
<?php
class BMDSystemhausBlogBridge extends BridgeAbstract
{
const MAINTAINER = 'cn-tools';
const NAME = 'BMD SYSTEMHAUS GesmbH';
const CACHE_TIMEOUT = 21600; //6h
const URI = 'https://www.bmd.com';
const DONATION_URI = 'https://paypal.me/cntools';
const DESCRIPTION = 'BMD Systemhaus - We make business easy';
const BMD_FAV_ICON = 'https://www.bmd.com/favicon.ico';
const ITEMSTYLE = [
'ilcr' => '<table width="100%"><tr><td style="vertical-align: top;">{data_img}</td><td style="vertical-align: top;">{data_content}</td></tr></table>',
'clir' => '<table width="100%"><tr><td style="vertical-align: top;">{data_content}</td><td style="vertical-align: top;">{data_img}</td></tr></table>',
'itcb' => '<div>{data_img}<br />{data_content}</div>',
'ctib' => '<div>{data_content}<br />{data_img}</div>',
'co' => '{data_content}',
'io' => '{data_img}'
];
const PARAMETERS = [
'Blog' => [
'country' => [
'name' => 'Country',
'type' => 'list',
'values' => [
'Österreich' => 'at',
'Deutschland' => 'de',
'Schweiz' => 'ch',
'Slovensko' => 'sk',
'Cesko' => 'cz',
'Hungary' => 'hu',
],
'defaultValue' => 'at',
],
'style' => [
'name' => 'Style',
'type' => 'list',
'values' => [
'Image left, content right' => 'ilcr',
'Content left, image right' => 'clir',
'Image top, content bottom' => 'itcb',
'Content top, image bottom' => 'ctib',
'Content only' => 'co',
'Image only' => 'io',
],
'defaultValue' => 'ilcr',
]
]
];
//-----------------------------------------------------
public function collectData()
{
// get website content
$html = getSimpleHTMLDOM($this->getURI());
// Convert relative links in HTML into absolute links
$html = defaultLinkTo($html, self::URI);
// Convert lazy-loading images and frames (video embeds) into static elements
$html = convertLazyLoading($html);
foreach ($html->find('div#bmdNewsList div#bmdNewsList-Item') as $element) {
$itemScope = $element->find('div[itemscope=itemscope]', 0);
$item = [];
// set base article data
$item['title'] = $this->getMetaItemPropContent($itemScope, 'headline');
$item['timestamp'] = strtotime($this->getMetaItemPropContent($itemScope, 'datePublished'));
$item['author'] = $this->getMetaItemPropContent($itemScope->find('div[itemprop=author]', 0), 'name');
// find article image
$imageTag = '';
$image = $element->find('div.mediaelement.mediaelement-image img', 0);
if ((!is_null($image)) and ($image->src != '')) {
$item['enclosures'] = [$image->src];
$imageTag = '<img src="' . $image->src . '"/>';
}
// begin with right style
$content = self::ITEMSTYLE[$this->getInput('style')];
// render placeholder
$content = str_replace('{data_content}', $this->getMetaItemPropContent($itemScope, 'description'), $content);
$content = str_replace('{data_img}', $imageTag, $content);
// set finished content
$item['content'] = $content;
// get link to article
$link = $element->find('div#bmdNewsList-Text div#bmdNewsList-Title a', 0);
if (!is_null($link)) {
$item['uri'] = $link->href;
}
// init categories
$categories = [];
$tmpOne = [];
$tmpTwo = [];
// search first categorie span
$catElem = $element->find('div#bmdNewsList-Text div#bmdNewsList-Category span.news-list-category', 0);
$txt = trim($catElem->innertext);
$tmpOne = explode('/', $txt);
// split by 2 spaces
foreach ($tmpOne as $tmpElem) {
$tmpElem = trim($tmpElem);
$tmpData = preg_split('/ /', $tmpElem);
$tmpTwo = array_merge($tmpTwo, $tmpData);
}
// split by tabulator
foreach ($tmpTwo as $tmpElem) {
$tmpElem = trim($tmpElem);
$tmpData = preg_split('/\t+/', $tmpElem);
$categories = array_merge($categories, $tmpData);
}
// trim each categorie entries
$categories = array_map('trim', $categories);
// remove empty entries
$categories = array_filter($categories, function ($value) {
return !is_null($value) && $value !== '';
});
// set categories
if (count($categories) > 0) {
$item['categories'] = $categories;
}
// add item
if (($item['title'] != '') and ($item['content'] != '') and ($item['uri'] != '')) {
$this->items[] = $item;
}
}
}
//-----------------------------------------------------
public function detectParameters($url)
{
try {
$parsedUrl = Url::fromString($url);
} catch (UrlException $e) {
return null;
}
if (!in_array($parsedUrl->getHost(), ['www.bmd.com', 'bmd.com'])) {
return null;
}
$lang = '';
// extract language from url
$path = explode('/', $parsedUrl->getPath());
if (count($path) > 1) {
$lang = $path[1];
// validate data
if ($this->getURIbyCountry($lang) == '') {
$lang = '';
}
}
// if no country available, find language by browser
if ($lang == '') {
$srvLanguages = explode(';', $_SERVER['HTTP_ACCEPT_LANGUAGE']);
if (count($srvLanguages) > 0) {
$languages = explode(',', $srvLanguages[0]);
if (count($languages) > 0) {
for ($i = 0; $i < count($languages); $i++) {
$langDetails = explode('-', $languages[$i]);
if (count($langDetails) > 1) {
$lang = $langDetails[1];
} else {
$lang = substr($srvLanguages[0], 0, 2);
}
// validate data
if ($this->getURIbyCountry($lang) == '') {
$lang = '';
}
if ($lang != '') {
break;
}
}
}
}
}
// if no URL found by language, use AT as default
if ($this->getURIbyCountry($lang) == '') {
$lang = 'at';
}
$params = [];
$params['country'] = strtolower($lang);
return $params;
}
//-----------------------------------------------------
public function getURI()
{
$country = $this->getInput('country') ?? '';
$lURI = $this->getURIbyCountry($country);
return $lURI != '' ? $lURI : parent::getURI();
}
//-----------------------------------------------------
public function getIcon()
{
return self::BMD_FAV_ICON;
}
//-----------------------------------------------------
private function getMetaItemPropContent($elem, $key)
{
if (($key != '') and (!is_null($elem))) {
$metaElem = $elem->find('meta[itemprop=' . $key . ']', 0);
if (!is_null($metaElem)) {
return $metaElem->getAttribute('content');
}
}
return '';
}
//-----------------------------------------------------
private function getURIbyCountry($country)
{
switch (strtolower($country)) {
case 'at':
return 'https://www.bmd.com/at/ueber-bmd/blog-ohne-filter.html';
case 'de':
return 'https://www.bmd.com/de/das-ist-bmd/blog.html';
case 'ch':
return 'https://www.bmd.com/ch/das-ist-bmd/blog.html';
case 'sk':
return 'https://www.bmd.com/sk/firma/blog.html';
case 'cz':
return 'https://www.bmd.com/cz/firma/news-blog.html';
case 'hu':
return 'https://www.bmd.com/hu/rolunk/hirek.html';
default:
return '';
}
}
}

441
bridges/BadDragonBridge.php Normal file
View File

@@ -0,0 +1,441 @@
<?php
class BadDragonBridge extends BridgeAbstract
{
const NAME = 'Bad Dragon Bridge';
const URI = 'https://bad-dragon.com/';
const CACHE_TIMEOUT = 300; // 5min
const DESCRIPTION = 'Returns sales or new clearance items';
const MAINTAINER = 'Roliga';
const PARAMETERS = [
'Sales' => [
],
'Clearance' => [
'ready_made' => [
'name' => 'Ready Made',
'type' => 'checkbox'
],
'flop' => [
'name' => 'Flops',
'type' => 'checkbox'
],
'skus' => [
'name' => 'Products',
'exampleValue' => 'chanceflared, crackers',
'title' => 'Comma separated list of product SKUs'
],
'onesize' => [
'name' => 'One-Size',
'type' => 'checkbox'
],
'mini' => [
'name' => 'Mini',
'type' => 'checkbox'
],
'small' => [
'name' => 'Small',
'type' => 'checkbox'
],
'medium' => [
'name' => 'Medium',
'type' => 'checkbox'
],
'large' => [
'name' => 'Large',
'type' => 'checkbox'
],
'extralarge' => [
'name' => 'Extra Large',
'type' => 'checkbox'
],
'category' => [
'name' => 'Category',
'type' => 'list',
'values' => [
'All' => 'all',
'Accessories' => 'accessories',
'Merchandise' => 'merchandise',
'Dildos' => 'insertable',
'Masturbators' => 'penetrable',
'Packers' => 'packer',
'Lil\' Squirts' => 'shooter',
'Lil\' Vibes' => 'vibrator',
'Wearables' => 'wearable'
],
'defaultValue' => 'all',
],
'soft' => [
'name' => 'Soft Firmness',
'type' => 'checkbox'
],
'med_firm' => [
'name' => 'Medium Firmness',
'type' => 'checkbox'
],
'firm' => [
'name' => 'Firm',
'type' => 'checkbox'
],
'split' => [
'name' => 'Split Firmness',
'type' => 'checkbox'
],
'maxprice' => [
'name' => 'Max Price',
'type' => 'number',
'required' => true,
'defaultValue' => 300
],
'minprice' => [
'name' => 'Min Price',
'type' => 'number',
'defaultValue' => 0
],
'cumtube' => [
'name' => 'Cumtube',
'type' => 'checkbox'
],
'suctionCup' => [
'name' => 'Suction Cup',
'type' => 'checkbox'
],
'noAccessories' => [
'name' => 'No Accessories',
'type' => 'checkbox'
]
]
];
/*
* This sets index $strFrom (or $strTo if set) in $outArr to 'on' if
* $inArr[$param] contains $strFrom.
* It is used for translating BD's shop filter URLs into something we can use.
*
* For the query '?type[]=ready_made&type[]=flop' we would have an array like:
* Array (
* [type] => Array (
* [0] => ready_made
* [1] => flop
* )
* )
* which could be translated into:
* Array (
* [ready_made] => on
* [flop] => on
* )
* */
private function setParam($inArr, &$outArr, $param, $strFrom, $strTo = null)
{
if (isset($inArr[$param]) && in_array($strFrom, $inArr[$param])) {
$outArr[($strTo ?: $strFrom)] = 'on';
}
}
public function detectParameters($url)
{
$params = [];
// Sale
$regex = '/^(https?:\/\/)?bad-dragon\.com\/sales/';
if (preg_match($regex, $url, $matches) > 0) {
$params['context'] = 'Sales';
return $params;
}
// Clearance
$regex = '/^(https?:\/\/)?bad-dragon\.com\/shop\/clearance/';
if (preg_match($regex, $url, $matches) > 0) {
parse_str(parse_url($url, PHP_URL_QUERY), $urlParams);
$this->setParam($urlParams, $params, 'type', 'ready_made');
$this->setParam($urlParams, $params, 'type', 'flop');
if (isset($urlParams['skus'])) {
$skus = [];
foreach ($urlParams['skus'] as $sku) {
is_string($sku) && $skus[] = $sku;
is_array($sku) && $skus[] = $sku[0];
}
$params['skus'] = implode(',', $skus);
}
$this->setParam($urlParams, $params, 'sizes', 'onesize');
$this->setParam($urlParams, $params, 'sizes', 'mini');
$this->setParam($urlParams, $params, 'sizes', 'small');
$this->setParam($urlParams, $params, 'sizes', 'medium');
$this->setParam($urlParams, $params, 'sizes', 'large');
$this->setParam($urlParams, $params, 'sizes', 'extralarge');
if (isset($urlParams['category'])) {
$params['category'] = strtolower($urlParams['category']);
} else {
$params['category'] = 'all';
}
$this->setParam($urlParams, $params, 'firmnessValues', 'soft');
$this->setParam($urlParams, $params, 'firmnessValues', 'medium', 'med_firm');
$this->setParam($urlParams, $params, 'firmnessValues', 'firm');
$this->setParam($urlParams, $params, 'firmnessValues', 'split');
if (isset($urlParams['price'])) {
isset($urlParams['price']['max'])
&& $params['maxprice'] = $urlParams['price']['max'];
isset($urlParams['price']['min'])
&& $params['minprice'] = $urlParams['price']['min'];
}
isset($urlParams['cumtube'])
&& $urlParams['cumtube'] === '1'
&& $params['cumtube'] = 'on';
isset($urlParams['suctionCup'])
&& $urlParams['suctionCup'] === '1'
&& $params['suctionCup'] = 'on';
isset($urlParams['noAccessories'])
&& $urlParams['noAccessories'] === '1'
&& $params['noAccessories'] = 'on';
$params['context'] = 'Clearance';
return $params;
}
return null;
}
public function getName()
{
switch ($this->queriedContext) {
case 'Sales':
return 'Bad Dragon Sales';
case 'Clearance':
return 'Bad Dragon Clearance Search';
default:
return parent::getName();
}
}
public function getURI()
{
switch ($this->queriedContext) {
case 'Sales':
return self::URI . 'sales';
case 'Clearance':
return $this->inputToURL();
default:
return parent::getURI();
}
}
public function collectData()
{
switch ($this->queriedContext) {
case 'Sales':
$sales = json_decode(getContents(self::URI . 'api/sales'));
foreach ($sales as $sale) {
$item = [];
$item['title'] = $sale->title;
$item['timestamp'] = strtotime($sale->startDate);
$item['uri'] = $this->getURI() . '/' . $sale->slug;
$contentHTML = '<p><img src="' . $sale->image->url . '"></p>';
if (isset($sale->endDate)) {
$contentHTML .= '<p><b>This promotion ends on '
. gmdate('M j, Y \a\t g:i A T', strtotime($sale->endDate))
. '</b></p>';
} else {
$contentHTML .= '<p><b>This promotion never ends</b></p>';
}
$ul = false;
$content = json_decode($sale->content);
foreach ($content->blocks as $block) {
switch ($block->type) {
case 'header-one':
$contentHTML .= '<h1>' . $block->text . '</h1>';
break;
case 'header-two':
$contentHTML .= '<h2>' . $block->text . '</h2>';
break;
case 'header-three':
$contentHTML .= '<h3>' . $block->text . '</h3>';
break;
case 'unordered-list-item':
if (!$ul) {
$contentHTML .= '<ul>';
$ul = true;
}
$contentHTML .= '<li>' . $block->text . '</li>';
break;
default:
if ($ul) {
$contentHTML .= '</ul>';
$ul = false;
}
$contentHTML .= '<p>' . $block->text . '</p>';
break;
}
}
$item['content'] = $contentHTML;
$this->items[] = $item;
}
break;
case 'Clearance':
$toyData = json_decode(getContents($this->inputToURL(true)));
$productList = json_decode(getContents(self::URI . 'api/inventory-toy/product-list'));
foreach ($toyData->toys as $toy) {
$item = [];
$item['uri'] = $this->getURI()
. '#'
. $toy->id;
$item['timestamp'] = strtotime($toy->created);
foreach ($productList as $product) {
if ($product->sku == $toy->sku) {
$item['title'] = $product->name;
break;
}
}
// images
$content = '<p>';
foreach ($toy->images as $image) {
$content .= '<a href="'
. $image->fullFilename
. '"><img src="'
. $image->thumbFilename
. '" /></a>';
}
// price
$content .= '</p><p><b>Price:</b> $'
. $toy->price
// size
. '<br /><b>Size:</b> '
. $toy->size
// color
. '<br /><b>Color:</b> '
. $toy->color
// features
. '<br /><b>Features:</b> '
. ($toy->suction_cup ? 'Suction cup' : '')
. ($toy->suction_cup && $toy->cumtube ? ', ' : '')
. ($toy->cumtube ? 'Cumtube' : '')
. ($toy->suction_cup || $toy->cumtube ? '' : 'None');
// firmness
$firmnessTexts = [
'2' => 'Extra soft',
'3' => 'Soft',
'5' => 'Medium',
'8' => 'Firm'
];
$firmnesses = explode('/', $toy->firmness);
if (count($firmnesses) === 2) {
$content .= '<br /><b>Firmness:</b> '
. $firmnessTexts[$firmnesses[0]]
. ', '
. $firmnessTexts[$firmnesses[1]];
} else {
$content .= '<br /><b>Firmness:</b> '
. $firmnessTexts[$firmnesses[0]];
}
// flop
if ($toy->type === 'flop') {
$content .= '<br /><b>Flop reason:</b> '
. $toy->flop_reason;
}
$content .= '</p>';
$item['content'] = $content;
$enclosures = [];
foreach ($toy->images as $image) {
$enclosures[] = $image->fullFilename;
}
$item['enclosures'] = $enclosures;
$categories = [];
$categories[] = $toy->sku;
$categories[] = $toy->type;
$categories[] = $toy->size;
if ($toy->cumtube) {
$categories[] = 'cumtube';
}
if ($toy->suction_cup) {
$categories[] = 'suction_cup';
}
$item['categories'] = $categories;
$this->items[] = $item;
}
break;
}
}
private function inputToURL($api = false)
{
$url = self::URI;
$url .= ($api ? 'api/inventory-toys?' : 'shop/clearance?');
// Default parameters
$url .= 'limit=60';
$url .= '&page=1';
$url .= '&sort[field]=created';
$url .= '&sort[direction]=desc';
// Product types
$url .= ($this->getInput('ready_made') ? '&type[]=ready_made' : '');
$url .= ($this->getInput('flop') ? '&type[]=flop' : '');
// Product names
foreach (array_filter(explode(',', $this->getInput('skus'))) as $sku) {
$url .= '&skus[]=' . urlencode(trim($sku));
}
// Size
$url .= ($this->getInput('onesize') ? '&sizes[]=onesize' : '');
$url .= ($this->getInput('mini') ? '&sizes[]=mini' : '');
$url .= ($this->getInput('small') ? '&sizes[]=small' : '');
$url .= ($this->getInput('medium') ? '&sizes[]=medium' : '');
$url .= ($this->getInput('large') ? '&sizes[]=large' : '');
$url .= ($this->getInput('extralarge') ? '&sizes[]=extralarge' : '');
// Category
$url .= ($this->getInput('category') ? '&category='
. urlencode($this->getInput('category')) : '');
// Firmness
if ($api) {
$url .= ($this->getInput('soft') ? '&firmnessValues[]=3' : '');
$url .= ($this->getInput('med_firm') ? '&firmnessValues[]=5' : '');
$url .= ($this->getInput('firm') ? '&firmnessValues[]=8' : '');
if ($this->getInput('split')) {
$url .= '&firmnessValues[]=3/5';
$url .= '&firmnessValues[]=3/8';
$url .= '&firmnessValues[]=8/3';
$url .= '&firmnessValues[]=5/8';
$url .= '&firmnessValues[]=8/5';
}
} else {
$url .= ($this->getInput('soft') ? '&firmnessValues[]=soft' : '');
$url .= ($this->getInput('med_firm') ? '&firmnessValues[]=medium' : '');
$url .= ($this->getInput('firm') ? '&firmnessValues[]=firm' : '');
$url .= ($this->getInput('split') ? '&firmnessValues[]=split' : '');
}
// Price
$url .= ($this->getInput('maxprice') ? '&price[max]='
. $this->getInput('maxprice') : '&price[max]=300');
$url .= ($this->getInput('minprice') ? '&price[min]='
. $this->getInput('minprice') : '&price[min]=0');
// Features
$url .= ($this->getInput('cumtube') ? '&cumtube=1' : '');
$url .= ($this->getInput('suctionCup') ? '&suctionCup=1' : '');
$url .= ($this->getInput('noAccessories') ? '&noAccessories=1' : '');
return $url;
}
}

View File

@@ -0,0 +1,211 @@
<?php
class BakaUpdatesMangaReleasesBridge extends BridgeAbstract
{
const NAME = 'Baka Updates Manga Releases';
const URI = 'https://www.mangaupdates.com/';
const DESCRIPTION = 'Get the latest series releases';
const MAINTAINER = 'fulmeek, KamaleiZestri';
const PARAMETERS = [
'By series' => [
'series_id' => [
'name' => 'Series ID',
'type' => 'number',
'required' => true,
'exampleValue' => '188066'
]
],
'By list' => [
'list_id' => [
'name' => 'List ID and Type',
'type' => 'text',
'required' => true,
'exampleValue' => '4395&list=read'
]
]
];
const LIMIT_COLS = 5;
const LIMIT_ITEMS = 10;
const RELEASES_URL = 'https://www.mangaupdates.com/releases.html';
private $feedName = '';
public function collectData()
{
if ($this -> queriedContext == 'By series') {
$this -> collectDataBySeries();
} else { //queriedContext == 'By list'
$this -> collectDataByList();
}
}
public function getURI()
{
if ($this -> queriedContext == 'By series') {
$series_id = $this->getInput('series_id');
if (!empty($series_id)) {
return self::URI . 'releases.html?search=' . $series_id . '&stype=series';
}
} else { //queriedContext == 'By list'
return self::RELEASES_URL;
}
return self::URI;
}
public function getName()
{
if (!empty($this->feedName)) {
return $this->feedName . ' - ' . self::NAME;
}
return parent::getName();
}
private function getSanitizedHash($string)
{
return hash('sha1', preg_replace('/[^a-zA-Z0-9\-\.]/', '', ucwords(strtolower($string))));
}
private function filterText($text)
{
return rtrim($text, '* ');
}
private function filterHTML($text)
{
return $this->filterText(html_entity_decode($text));
}
private function findID($manga)
{
// sometimes new series are on the release list that have no ID. just drop them.
if (@$this -> filterHTML($manga -> find('a', 0) -> href) != null) {
preg_match('/id=([0-9]*)/', $this -> filterHTML($manga -> find('a', 0) -> href), $match);
return $match[1];
} else {
return 0;
}
}
private function collectDataBySeries()
{
$html = getSimpleHTMLDOM($this->getURI());
// content is an unstructured pile of divs, ugly to parse
$cols = $html->find('div#main_content div.row > div.text');
if (!$cols) {
throwServerException('No releases');
}
$rows = array_slice(
array_chunk($cols, self::LIMIT_COLS),
0,
self::LIMIT_ITEMS
);
if (isset($rows[0][1])) {
$this->feedName = $this->filterHTML($rows[0][1]->plaintext);
}
foreach ($rows as $cols) {
if (count($cols) < self::LIMIT_COLS) {
continue;
}
$item = [];
$title = [];
$item['content'] = '';
$objDate = $cols[0];
if ($objDate) {
$item['timestamp'] = strtotime($objDate->plaintext);
}
$objTitle = $cols[1];
if ($objTitle) {
$title[] = $this->filterHTML($objTitle->plaintext);
$item['content'] .= '<p>Series: ' . $this->filterText($objTitle->innertext) . '</p>';
}
$objVolume = $cols[2];
if ($objVolume && !empty($objVolume->plaintext)) {
$title[] = 'Vol.' . $objVolume->plaintext;
}
$objChapter = $cols[3];
if ($objChapter && !empty($objChapter->plaintext)) {
$title[] = 'Chp.' . $objChapter->plaintext;
}
$objAuthor = $cols[4];
if ($objAuthor && !empty($objAuthor->plaintext)) {
$item['author'] = $this->filterHTML($objAuthor->plaintext);
$item['content'] .= '<p>Groups: ' . $this->filterText($objAuthor->innertext) . '</p>';
}
$item['title'] = implode(' ', $title);
$item['uri'] = $this->getURI();
$item['uid'] = $this->getSanitizedHash($item['title'] . $item['author']);
$this->items[] = $item;
}
}
private function collectDataByList()
{
$this -> feedName = 'Releases';
$list = [];
$releasesHTML = getSimpleHTMLDOM(self::RELEASES_URL);
$list_id = $this -> getInput('list_id');
$listHTML = getSimpleHTMLDOM('https://www.mangaupdates.com/mylist.html?id=' . $list_id);
//get ids of the manga that the user follows,
$parts = $listHTML -> find('table#ptable tr > td.pl');
foreach ($parts as $part) {
$list[] = $this -> findID($part);
}
//similar to above, but the divs are in groups of 3.
$cols = $releasesHTML -> find('div#main_content div.row > div.pbreak');
$rows = array_slice(array_chunk($cols, 3), 0);
foreach ($rows as $cols) {
//check if current manga is in user's list.
$id = $this -> findId($cols[0]);
if (!array_search($id, $list)) {
continue;
}
$item = [];
$title = [];
$item['content'] = '';
$objTitle = $cols[0];
if ($objTitle) {
$title[] = $this->filterHTML($objTitle->plaintext);
$item['content'] .= '<p>Series: ' . $this->filterHTML($objTitle -> innertext) . '</p>';
}
$objVolChap = $cols[1];
if ($objVolChap && !empty($objVolChap->plaintext)) {
$title[] = $this -> filterHTML($objVolChap -> innertext);
}
$objAuthor = $cols[2];
if ($objAuthor && !empty($objAuthor->plaintext)) {
$item['author'] = $this->filterHTML($objAuthor -> plaintext);
$item['content'] .= '<p>Groups: ' . $this->filterHTML($objAuthor -> innertext) . '</p>';
}
$item['title'] = implode(' ', $title);
$item['uri'] = self::URI . 'releases.html?search=' . $id . '&stype=series';
$item['uid'] = $this->getSanitizedHash($item['title'] . $item['author']);
$this->items[] = $item;
}
}
}

View File

@@ -1,63 +1,425 @@
<?php
class BandcampBridge extends BridgeAbstract {
const MAINTAINER = 'sebsauvage';
const NAME = 'Bandcamp Tag';
const URI = 'https://bandcamp.com/';
const CACHE_TIMEOUT = 600; // 10min
const DESCRIPTION = 'New bandcamp release by tag';
const PARAMETERS = array( array(
'tag' => array(
'name' => 'tag',
'type' => 'text',
'required' => true
)
));
class BandcampBridge extends BridgeAbstract
{
const MAINTAINER = 'sebsauvage, Roliga';
const NAME = 'Bandcamp Bridge';
const URI = 'https://bandcamp.com/';
const CACHE_TIMEOUT = 600; // 10min
const DESCRIPTION = 'New bandcamp releases by tag, band or album';
const PARAMETERS = [
'By tag' => [
'tag' => [
'name' => 'tag',
'type' => 'text',
'required' => true,
'exampleValue' => 'hip-hop-rap'
]
],
'By band' => [
'band' => [
'name' => 'band',
'type' => 'text',
'title' => 'Band name as seen in the band page URL',
'required' => true,
'exampleValue' => 'aesoprock'
],
'type' => [
'name' => 'Articles are',
'type' => 'list',
'values' => [
'Releases' => 'releases',
'Releases, new one when track list changes' => 'changes',
'Individual tracks' => 'tracks'
],
'defaultValue' => 'changes'
],
'limit' => [
'name' => 'limit',
'type' => 'number',
'required' => true,
'title' => 'Number of releases to return',
'defaultValue' => 5
]
],
'By label' => [
'label' => [
'name' => 'label',
'type' => 'text',
'title' => 'label name as seen in the label page URL',
'required' => true
],
'type' => [
'name' => 'Articles are',
'type' => 'list',
'values' => [
'Releases' => 'releases',
'Releases, new one when track list changes' => 'changes',
'Individual tracks' => 'tracks'
],
'defaultValue' => 'changes'
],
'limit' => [
'name' => 'limit',
'type' => 'number',
'title' => 'Number of releases to return',
'defaultValue' => 5
]
],
'By album' => [
'band' => [
'name' => 'band',
'type' => 'text',
'title' => 'Band name as seen in the album page URL',
'required' => true,
'exampleValue' => 'aesoprock'
],
'album' => [
'name' => 'album',
'type' => 'text',
'title' => 'Album name as seen in the album page URL',
'required' => true,
'exampleValue' => 'appleseed'
],
'type' => [
'name' => 'Articles are',
'type' => 'list',
'values' => [
'Releases' => 'releases',
'Releases, new one when track list changes' => 'changes',
'Individual tracks' => 'tracks'
],
'defaultValue' => 'tracks'
]
]
];
const IMGURI = 'https://f4.bcbits.com/';
const IMGSIZE_300PX = 23;
const IMGSIZE_700PX = 16;
public function collectData(){
$html = getSimpleHTMLDOM($this->getURI())
or returnServerError('No results for this query.');
private $feedName;
foreach($html->find('li.item') as $release) {
$script = $release->find('div.art', 0)->getAttribute('onclick');
$uri = ltrim($script, "return 'url(");
$uri = rtrim($uri, "')");
public function getIcon()
{
return 'https://s4.bcbits.com/img/bc_favicon.ico';
}
$item = array();
$item['author'] = $release->find('div.itemsubtext', 0)->plaintext
. ' - '
. $release->find('div.itemtext', 0)->plaintext;
public function collectData()
{
switch ($this->queriedContext) {
case 'By tag':
$url = self::URI . 'api/hub/1/dig_deeper';
$data = $this->buildRequestJson();
$header = [
'Content-Type: application/json',
'Content-Length: ' . strlen($data),
];
$opts = [
CURLOPT_CUSTOMREQUEST => 'POST',
CURLOPT_POSTFIELDS => $data,
];
$content = getContents($url, $header, $opts);
$item['title'] = $release->find('div.itemsubtext', 0)->plaintext
. ' - '
. $release->find('div.itemtext', 0)->plaintext;
$json = json_decode($content);
$item['content'] = '<img src="'
. $uri
. '"/><br/>'
. $release->find('div.itemsubtext', 0)->plaintext
. ' - '
. $release->find('div.itemtext', 0)->plaintext;
if ($json->ok !== true) {
throwServerException('Invalid response');
}
$item['id'] = $release->find('a', 0)->getAttribute('href');
$item['uri'] = $release->find('a', 0)->getAttribute('href');
$this->items[] = $item;
}
}
foreach ($json->items as $entry) {
$url = $entry->tralbum_url;
$artist = $entry->artist;
$title = $entry->title;
// e.g. record label is the releaser, but not the artist
$releaser = $entry->band_name !== $entry->artist ? $entry->band_name : null;
public function getURI(){
if(!is_null($this->getInput('tag'))) {
return self::URI . 'tag/' . urlencode($this->getInput('tag')) . '?sort_field=date';
}
$full_title = $artist . ' - ' . $title;
$full_artist = $artist;
if (isset($releaser)) {
$full_title .= ' (' . $releaser . ')';
$full_artist .= ' (' . $releaser . ')';
}
$small_img = $this->getImageUrl($entry->art_id, self::IMGSIZE_300PX);
$img = $this->getImageUrl($entry->art_id, self::IMGSIZE_700PX);
return parent::getURI();
}
$item = [
'uri' => $url,
'author' => $full_artist,
'title' => $full_title
];
$item['content'] = "<img src='$small_img' /><br/>$full_title";
$item['enclosures'] = [$img];
$this->items[] = $item;
}
break;
case 'By band':
case 'By label':
case 'By album':
$html = getSimpleHTMLDOMCached($this->getURI(), 86400);
public function getName(){
if(!is_null($this->getInput('tag'))) {
return $this->getInput('tag') . ' - Bandcamp Tag';
}
if ($html->find('meta[name=title]', 0)) {
$this->feedName = $html->find('meta[name=title]', 0)->content;
} else {
$this->feedName = str_replace('Music | ', '', $html->find('title', 0)->plaintext);
}
return parent::getName();
}
$regex = '/band_id=(\d+)/';
if (preg_match($regex, $html, $matches) == false) {
throwServerException('Unable to find band ID on: ' . $this->getURI());
}
$band_id = $matches[1];
$tralbums = [];
switch ($this->queriedContext) {
case 'By band':
case 'By label':
$query_data = [
'band_id' => $band_id
];
$band_data = $this->apiGet('mobile/22/band_details', $query_data);
$num_albums = min(count($band_data->discography), $this->getInput('limit'));
for ($i = 0; $i < $num_albums; $i++) {
$album_basic_data = $band_data->discography[$i];
// 'a' or 't' for albums and individual tracks respectively
$tralbum_type = substr($album_basic_data->item_type, 0, 1);
$query_data = [
'band_id' => $band_id,
'tralbum_type' => $tralbum_type,
'tralbum_id' => $album_basic_data->item_id
];
$tralbums[] = $this->apiGet('mobile/22/tralbum_details', $query_data);
}
break;
case 'By album':
$regex = '/album=(\d+)/';
if (preg_match($regex, $html, $matches) == false) {
throwServerException('Unable to find album ID on: ' . $this->getURI());
}
$album_id = $matches[1];
$query_data = [
'band_id' => $band_id,
'tralbum_type' => 'a',
'tralbum_id' => $album_id
];
$tralbums[] = $this->apiGet('mobile/22/tralbum_details', $query_data);
break;
}
foreach ($tralbums as $tralbum_data) {
if ($tralbum_data->type === 'a' && $this->getInput('type') === 'tracks') {
foreach ($tralbum_data->tracks as $track) {
$query_data = [
'band_id' => $band_id,
'tralbum_type' => 't',
'tralbum_id' => $track->track_id
];
$track_data = $this->apiGet('mobile/22/tralbum_details', $query_data);
$this->items[] = $this->buildTralbumItem($track_data);
}
} else {
$this->items[] = $this->buildTralbumItem($tralbum_data);
}
}
break;
}
}
private function buildTralbumItem($tralbum_data)
{
$band_data = $tralbum_data->band;
// Format title like: ARTIST - ALBUM/TRACK (OPTIONAL RELEASER)
// Format artist/author like: ARTIST (OPTIONAL RELEASER)
//
// If the album/track is released under a label/a band other than the artist
// themselves, append that releaser name to the title and artist/author.
//
// This sadly doesn't always work right for individual tracks as the artist
// of the track is always set to the releaser.
$artist = $tralbum_data->tralbum_artist;
$full_title = $artist . ' - ' . $tralbum_data->title;
$full_artist = $artist;
if (isset($tralbum_data->label)) {
$full_title .= ' (' . $tralbum_data->label . ')';
$full_artist .= ' (' . $tralbum_data->label . ')';
} elseif ($band_data->name !== $artist) {
$full_title .= ' (' . $band_data->name . ')';
$full_artist .= ' (' . $band_data->name . ')';
}
$small_img = $this->getImageUrl($tralbum_data->art_id, self::IMGSIZE_300PX);
$img = $this->getImageUrl($tralbum_data->art_id, self::IMGSIZE_700PX);
$item = [
'uri' => $tralbum_data->bandcamp_url,
'author' => $full_artist,
'title' => $full_title,
'enclosures' => [$img],
'timestamp' => $tralbum_data->release_date
];
$item['categories'] = [];
foreach ($tralbum_data->tags as $tag) {
$item['categories'][] = $tag->norm_name;
}
// Give articles a unique UID depending on its track list
// Releases should then show up as new articles when tracks are added
if ($this->getInput('type') === 'changes') {
$item['uid'] = "bandcamp/$band_data->band_id/$tralbum_data->id/";
foreach ($tralbum_data->tracks as $track) {
$item['uid'] .= $track->track_id;
}
}
$item['content'] = "<img src='$small_img' /><br/>$full_title<br/>";
if ($tralbum_data->type === 'a') {
$item['content'] .= '<ol>';
foreach ($tralbum_data->tracks as $track) {
$item['content'] .= "<li>$track->title</li>";
}
$item['content'] .= '</ol>';
}
if (!empty($tralbum_data->about)) {
$item['content'] .= '<p>'
. nl2br($tralbum_data->about)
. '</p>';
}
return $item;
}
private function buildRequestJson()
{
$requestJson = [
'tag' => $this->getInput('tag'),
'page' => 1,
'sort' => 'date'
];
return json_encode($requestJson);
}
private function getImageUrl($id, $size)
{
return self::IMGURI . 'img/a' . $id . '_' . $size . '.jpg';
}
private function apiGet($endpoint, $query_data)
{
$url = self::URI . 'api/' . $endpoint . '?' . http_build_query($query_data);
// todo: 429 Too Many Requests happens a lot
$response = getContents($url);
$data = json_decode($response);
return $data;
}
public function getURI()
{
switch ($this->queriedContext) {
case 'By tag':
if (!is_null($this->getInput('tag'))) {
return self::URI
. 'tag/'
. urlencode($this->getInput('tag'))
. '?sort_field=date';
}
break;
case 'By label':
if (!is_null($this->getInput('label'))) {
return 'https://'
. $this->getInput('label')
. '.bandcamp.com/music';
}
break;
case 'By band':
if (!is_null($this->getInput('band'))) {
return 'https://'
. $this->getInput('band')
. '.bandcamp.com/music';
}
break;
case 'By album':
if (!is_null($this->getInput('band')) && !is_null($this->getInput('album'))) {
return 'https://'
. $this->getInput('band')
. '.bandcamp.com/album/'
. $this->getInput('album');
}
break;
}
return parent::getURI();
}
public function getName()
{
switch ($this->queriedContext) {
case 'By tag':
if (!is_null($this->getInput('tag'))) {
return $this->getInput('tag') . ' - Bandcamp Tag';
}
break;
case 'By band':
if (isset($this->feedName)) {
return $this->feedName . ' - Bandcamp Band';
} elseif (!is_null($this->getInput('band'))) {
return $this->getInput('band') . ' - Bandcamp Band';
}
break;
case 'By label':
if (isset($this->feedName)) {
return $this->feedName . ' - Bandcamp Label';
} elseif (!is_null($this->getInput('label'))) {
return $this->getInput('label') . ' - Bandcamp Label';
}
break;
case 'By album':
if (isset($this->feedName)) {
return $this->feedName . ' - Bandcamp Album';
} elseif (!is_null($this->getInput('album'))) {
return $this->getInput('album') . ' - Bandcamp Album';
}
break;
}
return parent::getName();
}
public function detectParameters($url)
{
$params = [];
// By tag
$regex = '/^(https?:\/\/)?bandcamp\.com\/tag\/([^\/.&?\n]+)/';
if (preg_match($regex, $url, $matches) > 0) {
$params['context'] = 'By tag';
$params['tag'] = urldecode($matches[2]);
return $params;
}
// By band
$regex = '/^(https?:\/\/)?([^\/.&?\n]+?)\.bandcamp\.com/';
if (preg_match($regex, $url, $matches) > 0) {
$params['context'] = 'By band';
$params['band'] = urldecode($matches[2]);
return $params;
}
// By album
$regex = '/^(https?:\/\/)?([^\/.&?\n]+?)\.bandcamp\.com\/album\/([^\/.&?\n]+)/';
if (preg_match($regex, $url, $matches) > 0) {
$params['context'] = 'By album';
$params['band'] = urldecode($matches[2]);
$params['album'] = urldecode($matches[3]);
return $params;
}
return null;
}
}

View File

@@ -0,0 +1,162 @@
<?php
class BandcampDailyBridge extends BridgeAbstract
{
const NAME = 'Bandcamp Daily Bridge';
const URI = 'https://daily.bandcamp.com';
const DESCRIPTION = 'Returns newest articles';
const MAINTAINER = 'VerifiedJoseph';
const PARAMETERS = [
'Latest articles' => [],
'Best of' => [
'best-content' => [
'name' => 'content',
'type' => 'list',
'values' => [
'Best Ambient' => 'best-ambient',
'Best Beat Tapes' => 'best-beat-tapes',
'Best Dance 12\'s' => 'best-dance-12s',
'Best Contemporary Classical' => 'best-contemporary-classical',
'Best Electronic' => 'best-electronic',
'Best Experimental' => 'best-experimental',
'Best Hip-Hop' => 'best-hip-hop',
'Best Jazz' => 'best-jazz',
'Best Metal' => 'best-metal',
'Best Punk' => 'best-punk',
'Best Reissues' => 'best-reissues',
'Best Soul' => 'best-soul',
],
'defaultValue' => 'best-ambient',
],
],
'Genres' => [
'genres-content' => [
'name' => 'content',
'type' => 'list',
'values' => [
'Acoustic' => 'genres/acoustic',
'Alternative' => 'genres/alternative',
'Ambient' => 'genres/ambient',
'Blues' => 'genres/blues',
'Classical' => 'genres/classical',
'Comedy' => 'genres/comedy',
'Country' => 'genres/country',
'Devotional' => 'genres/devotional',
'Electronic' => 'genres/electronic',
'Experimental' => 'genres/experimental',
'Folk' => 'genres/folk',
'Funk' => 'genres/funk',
'Hip-Hop/Rap' => 'genres/hip-hop-rap',
'Jazz' => 'genres/jazz',
'Kids' => 'genres/kids',
'Latin' => 'genres/latin',
'Metal' => 'genres/metal',
'Pop' => 'genres/pop',
'Punk' => 'genres/punk',
'R&B/Soul' => 'genres/r-b-soul',
'Reggae' => 'genres/reggae',
'Rock' => 'genres/rock',
'Soundtrack' => 'genres/soundtrack',
'Spoken Word' => 'genres/spoken-word',
'World' => 'genres/world',
],
'defaultValue' => 'genres/acoustic',
],
],
'Franchises' => [
'franchises-content' => [
'name' => 'content',
'type' => 'list',
'values' => [
'Lists' => 'lists',
'Features' => 'features',
'Album of the Day' => 'album-of-the-day',
'Acid Test' => 'acid-test',
'Bandcamp Navigator' => 'bandcamp-navigator',
'Big Ups' => 'big-ups',
'Certified' => 'certified',
'Gallery' => 'gallery',
'Hidden Gems' => 'hidden-gems',
'High Scores' => 'high-scores',
'Label Profile' => 'label-profile',
'Lifetime Achievement' => 'lifetime-achievement',
'Scene Report' => 'scene-report',
'Seven Essential Releases' => 'seven-essential-releases',
'The Merch Table' => 'the-merch-table',
],
'defaultValue' => 'lists',
],
]
];
const CACHE_TIMEOUT = 3600; // 1 hour
public function collectData()
{
$html = getSimpleHTMLDOM($this->getURI());
$html = defaultLinkTo($html, self::URI);
$articles = $html->find('articles-list', 0);
foreach ($articles->find('div.list-article') as $index => $article) {
$item = [];
$articlePath = $article->find('a.title', 0)->href;
$articlePageHtml = getSimpleHTMLDOMCached($articlePath, 3600);
$item['uri'] = $articlePath;
$item['title'] = $articlePageHtml->find('article-title', 0)->innertext;
$item['author'] = $articlePageHtml->find('article-credits > a', 0)->innertext;
$item['content'] = html_entity_decode($articlePageHtml->find('meta[name="description"]', 0)->content, ENT_QUOTES);
$item['timestamp'] = $articlePageHtml->find('meta[property="article:published_time"]', 0)->content;
$item['categories'][] = $articlePageHtml->find('meta[property="article:section"]', 0)->content;
if ($articlePageHtml->find('meta[property="article:tag"]', 0)) {
$item['categories'][] = $articlePageHtml->find('meta[property="article:tag"]', 0)->content;
}
$item['enclosures'][] = $articlePageHtml->find('meta[name="twitter:image"]', 0)->content;
$this->items[] = $item;
if (count($this->items) >= 10) {
break;
}
}
}
public function getURI()
{
switch ($this->queriedContext) {
case 'Latest articles':
return self::URI . '/latest';
case 'Best of':
case 'Genres':
case 'Franchises':
// TODO Switch to array_key_first once php >= 7.3
$contentKey = key(self::PARAMETERS[$this->queriedContext]);
return self::URI . '/' . $this->getInput($contentKey);
default:
return parent::getURI();
}
}
public function getName()
{
switch ($this->queriedContext) {
case 'Latest articles':
return $this->queriedContext . ' - Bandcamp Daily';
case 'Best of':
case 'Genres':
case 'Franchises':
$contentKey = array_key_first(self::PARAMETERS[$this->queriedContext]);
$contentValues = array_flip(self::PARAMETERS[$this->queriedContext][$contentKey]['values']);
return $contentValues[$this->getInput($contentKey)] . ' - Bandcamp Daily';
default:
return parent::getName();
}
}
}

View File

@@ -1,34 +1,33 @@
<?php
class BastaBridge extends BridgeAbstract {
const MAINTAINER = 'qwertygc';
const NAME = 'Bastamag Bridge';
const URI = 'http://www.bastamag.net/';
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = 'Returns the newest articles.';
class BastaBridge extends BridgeAbstract
{
const MAINTAINER = 'qwertygc';
const NAME = 'Bastamag Bridge';
const URI = 'https://www.bastamag.net/';
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = 'Returns the newest articles.';
public function collectData(){
// Replaces all relative image URLs by absolute URLs.
// Relative URLs always start with 'local/'!
function replaceImageUrl($content){
return preg_replace('/src=["\']{1}([^"\']+)/ims', 'src=\'' . self::URI . '$1\'', $content);
}
public function collectData()
{
$html = getSimpleHTMLDOM(self::URI . 'spip.php?page=backend');
$html = getSimpleHTMLDOM(self::URI . 'spip.php?page=backend')
or returnServerError('Could not request Bastamag.');
$limit = 0;
$limit = 0;
foreach ($html->find('item') as $element) {
if ($limit < 10) {
$item = [];
$item['title'] = $element->find('title', 0)->innertext;
$item['uri'] = $element->find('guid', 0)->plaintext;
$item['timestamp'] = strtotime($element->find('dc:date', 0)->plaintext);
foreach($html->find('item') as $element) {
if($limit < 10) {
$item = array();
$item['title'] = $element->find('title', 0)->innertext;
$item['uri'] = $element->find('guid', 0)->plaintext;
$item['timestamp'] = strtotime($element->find('dc:date', 0)->plaintext);
$item['content'] = replaceImageUrl(getSimpleHTMLDOM($item['uri'])->find('div.texte', 0)->innertext);
$this->items[] = $item;
$limit++;
}
}
}
$html = getSimpleHTMLDOM($item['uri']);
$html = defaultLinkTo($html, self::URI);
$item['content'] = $html->find('div.texte', 0)->innertext;
$this->items[] = $item;
$limit++;
}
}
}
}

139
bridges/BazarakiBridge.php Normal file
View File

@@ -0,0 +1,139 @@
<?php
class BazarakiBridge extends BridgeAbstract
{
const NAME = 'Bazaraki Bridge';
const URI = 'https://bazaraki.com';
const DESCRIPTION = 'Fetch adverts from Bazaraki, a Cyprus-based classifieds website.';
const MAINTAINER = 'danwain';
const PARAMETERS = [
[
'url' => [
'name' => 'URL',
'type' => 'text',
'required' => true,
'title' => 'Enter the URL of the Bazaraki page to fetch adverts from.',
'exampleValue' => 'https://www.bazaraki.com/real-estate-for-sale/houses/?lat=0&lng=0&radius=100000',
],
'limit' => [
'name' => 'Limit',
'type' => 'number',
'required' => false,
'title' => 'Enter the number of adverts to fetch. (max 50)',
'exampleValue' => '10',
'defaultValue' => 10,
]
]
];
public function collectData()
{
$url = $this->getInput('url');
if (! str_starts_with($url, 'https://www.bazaraki.com/')) {
throw new \Exception('Nope');
}
$html = getSimpleHTMLDOM($url);
$i = 0;
foreach ($html->find('div.advert') as $element) {
$i++;
if ($i > $this->getInput('limit') || $i > 50) {
break;
}
$item = [];
$item['uri'] = 'https://www.bazaraki.com' . $element->find('a.advert__content-title', 0)->href;
# Get the content
$advert = getSimpleHTMLDOM($item['uri']);
$price = trim($advert->find('div.announcement-price__cost', 0)->plaintext);
$name = trim($element->find('a.advert__content-title', 0)->plaintext);
$item['title'] = $name . ' - ' . $price;
$time = trim($advert->find('span.date-meta', 0)->plaintext);
$time = str_replace('Posted: ', '', $time);
$item['content'] = $this->processAdvertContent($advert);
$item['timestamp'] = $this->convertRelativeTime($time);
$item['author'] = trim($advert->find('div.author-name', 0)->plaintext);
$item['uid'] = $advert->find('span.number-announcement', 0)->plaintext;
$this->items[] = $item;
}
}
/**
* Process the advert content to clean up HTML
*
* @param simple_html_dom $advert The SimpleHTMLDOM object for the advert page
* @return string Processed HTML content
*/
private function processAdvertContent($advert)
{
// Get the content sections
$header = $advert->find('div.announcement-content-header', 0);
$characteristics = $advert->find('div.announcement-characteristics', 0);
$description = $advert->find('div.js-description', 0);
$images = $advert->find('div.announcement__images', 0);
// Remove all favorites divs
foreach ($advert->find('div.announcement-meta__favorites') as $favorites) {
$favorites->outertext = '';
}
// Replace all <a> tags with their text content
foreach ($advert->find('a') as $a) {
$a->outertext = $a->innertext;
}
// Format the content with section headers and dividers
$formattedContent = '';
// Add header section
$formattedContent .= $header->innertext;
$formattedContent .= '<hr/>';
// Add characteristics section with header
$formattedContent .= '<h3>Details</h3>';
$formattedContent .= $characteristics->innertext;
$formattedContent .= '<hr/>';
// Add description section with header
$formattedContent .= '<h3>Description</h3>';
$formattedContent .= $description->innertext;
$formattedContent .= '<hr/>';
// Add images section with header
$formattedContent .= '<h3>Images</h3>';
$formattedContent .= $images->innertext;
return $formattedContent;
}
/**
* Convert relative time strings like "Yesterday 12:32" to proper timestamps
*
* @param string $timeString The relative time string from the website
* @return string Timestamp in a format compatible with strtotime()
*/
private function convertRelativeTime($timeString)
{
if (strpos($timeString, 'Yesterday') !== false) {
// Replace "Yesterday" with actual date
$time = str_replace('Yesterday', date('Y-m-d', strtotime('-1 day')), $timeString);
return date('Y-m-d H:i:s', strtotime($time));
} elseif (strpos($timeString, 'Today') !== false) {
// Replace "Today" with actual date
$time = str_replace('Today', date('Y-m-d'), $timeString);
return date('Y-m-d H:i:s', strtotime($time));
} else {
// For other formats, return as is and let strtotime handle it
return $timeString;
}
}
}

34
bridges/BinanceBridge.php Normal file
View File

@@ -0,0 +1,34 @@
<?php
class BinanceBridge extends BridgeAbstract
{
const NAME = 'Binance Blog';
const URI = 'https://www.binance.com/en/blog';
const DESCRIPTION = 'Subscribe to the Binance blog.';
const MAINTAINER = 'thefranke';
const CACHE_TIMEOUT = 3600; // 1h
public function collectData()
{
$url = 'https://www.binance.com/bapi/composite/v1/public/content/blog/list?category=&tag=&page=1&size=12';
$json = getContents($url);
$data = Json::decode($json, false);
foreach ($data->data->blogList as $post) {
$item = [];
$item['title'] = $post->title;
// Url slug not in json
//$item['uri'] = $uri;
$item['timestamp'] = $post->postTimeUTC / 1000;
$item['author'] = 'Binance';
$item['content'] = $post->brief;
//$item['categories'] = $category;
$item['uid'] = $post->idStr;
$this->items[] = $item;
}
}
public function getIcon()
{
return 'https://bin.bnbstatic.com/static/images/common/favicon.ico';
}
}

View File

@@ -1,31 +1,44 @@
<?php
class BlaguesDeMerdeBridge extends BridgeAbstract {
const MAINTAINER = 'superbaillot.net';
const NAME = 'Blagues De Merde';
const URI = 'http://www.blaguesdemerde.fr/';
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = 'Blagues De Merde';
class BlaguesDeMerdeBridge extends BridgeAbstract
{
const MAINTAINER = 'superbaillot.net, logmanoriginal';
const NAME = 'Blagues De Merde';
const URI = 'http://www.blaguesdemerde.fr/';
const CACHE_TIMEOUT = 7200; // 2h
const DESCRIPTION = 'Blagues De Merde';
public function collectData(){
$html = getSimpleHTMLDOM(self::URI)
or returnServerError('Could not request BDM.');
public function getIcon()
{
return self::URI . 'assets/img/favicon.ico';
}
foreach($html->find('article.joke_contener') as $element) {
$item = array();
$temp = $element->find('a');
public function collectData()
{
$html = getSimpleHTMLDOM(self::URI);
if(isset($temp[2])) {
$item['content'] = trim($element->find('div.joke_text_contener', 0)->innertext);
$uri = $temp[2]->href;
$item['uri'] = $uri;
$item['title'] = substr($uri, (strrpos($uri, "/") + 1));
$date = $element->find('li.bdm_date', 0)->innertext;
$time = mktime(0, 0, 0, substr($date, 3, 2), substr($date, 0, 2), substr($date, 6, 4));
$item['timestamp'] = $time;
$item['author'] = $element->find('li.bdm_pseudo', 0)->innertext;
$this->items[] = $item;
}
}
}
foreach ($html->find('div.blague') as $element) {
$item = [];
$item['uri'] = static::URI . '#' . $element->id;
$item['author'] = $element->find('div[class="blague-footer"] p strong', 0)->plaintext;
// Let the title be everything up to the first <br>
$item['title'] = trim(explode("\n", $element->find('div.text', 0)->plaintext)[0]);
$item['content'] = strip_tags($element->find('div.text', 0));
// timestamp is part of:
// <p>Par <strong>{author}</strong> le {date} dans <strong>{category}</strong></p>
preg_match(
'/.+le(.+)dans.*/',
$element->find('div[class="blague-footer"]', 0)->plaintext,
$matches
);
$item['timestamp'] = strtotime($matches[1]);
$this->items[] = $item;
}
}
}

View File

@@ -0,0 +1,30 @@
<?php
class BleepingComputerBridge extends FeedExpander
{
const MAINTAINER = 'csisoap';
const NAME = 'Bleeping Computer';
const URI = 'https://www.bleepingcomputer.com/';
const DESCRIPTION = 'Returns the newest articles.';
public function collectData()
{
$feed = static::URI . 'feed/';
$this->collectExpandableDatas($feed);
}
protected function parseItem(array $item)
{
$article_html = getSimpleHTMLDOMCached($item['uri']);
if (!$article_html) {
$item['content'] .= '<p><em>Could not request ' . $this->getName() . ': ' . $item['uri'] . '</em></p>';
return $item;
}
$article_content = $article_html->find('div.articleBody', 0)->innertext;
$article_content = stripRecursiveHTMLSection($article_content, 'div', '<div class="cz-related-article-wrapp');
$item['content'] = trim($article_content);
return $item;
}
}

View File

@@ -0,0 +1,107 @@
<?php
class BlizzardNewsBridge extends BridgeAbstract
{
const NAME = 'Blizzard News';
const URI = 'https://news.blizzard.com';
const DESCRIPTION = 'Blizzard (game company) newsfeed';
const MAINTAINER = 'Niehztog';
const PARAMETERS = [
'' => [
'locale' => [
'name' => 'Language',
'type' => 'list',
'values' => [
'Deutsch' => 'de-de',
'English (EU)' => 'en-gb',
'English (US)' => 'en-us',
'Español (EU)' => 'es-es',
'Español (AL)' => 'es-mx',
'Français' => 'fr-fr',
'Italiano' => 'it-it',
'日本語' => 'ja-jp',
'한국어' => 'ko-kr',
'Polski' => 'pl-pl',
'Português (AL)' => 'pt-br',
'Русский' => 'ru-ru',
'ภาษาไทย' => 'th-th',
'简体中文' => 'zh-cn',
'繁體中文' => 'zh-tw'
],
'defaultValue' => 'en-us',
'title' => 'Select your language'
]
]
];
const CACHE_TIMEOUT = 3600;
private const PRODUCT_IDS = [
'blt525c436e4a1b0a97',
'blt54fbd3787a705054',
'blt2031aef34200656d',
'blt795c314400d7ded9',
'blt5cfc6affa3ca0638',
'blt2e50e1521bb84dc6',
'blt376fb94931906b6f',
'blt81d46fcb05ab8811',
'bltede2389c0a8885aa',
'blt24859ba8086fb294',
'blte27d02816a8ff3e1',
'blt2caca37e42f19839',
'blt90855744d00cd378',
'bltec70ad0ea4fd6d1d',
'blt500c1f8b5470bfdb'
];
private const API_PATH = '/api/news/blizzard?';
/**
* Source Web page URL (should provide either HTML or XML content)
* @return string
*/
private function getSourceUrl(): string
{
$locale = $this->getInput('locale');
if ('zh-cn' === $locale) {
$baseUrl = 'https://cn.news.blizzard.com' . self::API_PATH;
} else {
$baseUrl = 'https://news.blizzard.com/' . $locale . self::API_PATH;
}
return $baseUrl .= http_build_query([
'feedCxpProductIds' => self::PRODUCT_IDS
]);
}
public function collectData()
{
$feedContent = json_decode(getContents($this->getSourceUrl()), true);
foreach ($feedContent['feed']['contentItems'] as $entry) {
$properties = $entry['properties'];
$item = [];
$item['title'] = $this->filterChars($properties['title']);
$item['content'] = $this->filterChars($properties['summary']);
$item['uri'] = $properties['newsUrl'];
$item['author'] = $this->filterChars($properties['author']);
$item['timestamp'] = strtotime($properties['lastUpdated']);
$item['enclosures'] = [$properties['staticAsset']['imageUrl']];
$item['categories'] = [$this->filterChars($properties['cxpProduct']['title'])];
$this->items[] = $item;
}
}
private function filterChars($content)
{
return htmlspecialchars($content, ENT_XML1);
}
public function getIcon()
{
return <<<icon
https://dfbmfbnnydoln.cloudfront.net/production/images/favicons/favicon.ba01bb119359d74970b02902472fd82e96b5aba7.ico
icon;
}
}

View File

@@ -1,65 +0,0 @@
<?php
class BloombergBridge extends BridgeAbstract
{
const NAME = 'Bloomberg';
const URI = 'https://www.bloomberg.com/';
const DESCRIPTION = 'Trending stories from Bloomberg';
const MAINTAINER = 'mdemoss';
const PARAMETERS = array(
'Trending Stories' => array(),
'From Search' => array(
'q' => array(
'name' => 'Keyword',
'required' => true
)
)
);
public function getName()
{
switch($this->queriedContext) {
case 'Trending Stories':
return self::NAME . ' Trending Stories';
case 'From Search':
if (!is_null($this->getInput('q'))) {
return self::NAME . ' Search : ' . $this->getInput('q');
}
break;
}
return parent::getName();
}
public function collectData()
{
switch($this->queriedContext) {
case 'Trending Stories': // Get list of top new <article>s from the front page.
$html = getSimpleHTMLDOMCached($this->getURI(), 300);
$stories = $html->find('ul.top-news-v3__stories article.top-news-v3-story');
break;
case 'From Search': // Get list of <article> elements from search.
$html = getSimpleHTMLDOMCached(
$this->getURI() .
'search?sort=time:desc&page=1&query=' .
urlencode($this->getInput('q')), 300
);
$stories = $html->find('div.search-result-items article.search-result-story');
break;
}
foreach ($stories as $element) {
$item['uri'] = $element->find('h1 a', 0)->href;
if (preg_match('#^https://#i', $item['uri']) !== 1) {
$item['uri'] = $this->getURI() . $item['uri'];
}
$articleHtml = getSimpleHTMLDOMCached($item['uri']);
if (!$articleHtml) {
continue;
}
$item['title'] = $element->find('h1 a', 0)->plaintext;
$item['timestamp'] = strtotime($articleHtml->find('meta[name=iso-8601-publish-date],meta[name=date]', 0)->content);
$item['content'] = $articleHtml->find('meta[name=description]', 0)->content;
$this->items[] = $item;
}
}
}

674
bridges/BlueskyBridge.php Normal file
View File

@@ -0,0 +1,674 @@
<?php
class BlueskyBridge extends BridgeAbstract
{
//Initial PR by [RSSBridge contributors](https://github.com/RSS-Bridge/rss-bridge/issues/4058).
//Modified from [©DIYgod and contributors at RSSHub](https://github.com/DIYgod/RSSHub/tree/master/lib/routes/bsky), MIT License';
const NAME = 'Bluesky Bridge';
const URI = 'https://bsky.app';
const DESCRIPTION = 'Fetches posts from Bluesky';
const MAINTAINER = 'mruac';
const PARAMETERS = [
[
'data_source' => [
'name' => 'Bluesky Data Source',
'type' => 'list',
'defaultValue' => 'Profile',
'values' => [
'Profile' => 'getAuthorFeed',
],
'title' => 'Select the type of data source to fetch from Bluesky.'
],
'user_id' => [
'name' => 'User Handle or DID',
'type' => 'text',
'required' => true,
'exampleValue' => 'did:plc:z72i7hdynmk6r22z27h6tvur',
'title' => 'ATProto / Bsky.app handle or DID'
],
'feed_filter' => [
'name' => 'Feed type',
'type' => 'list',
'defaultValue' => 'posts_and_author_threads',
'values' => [
'Posts feed' => 'posts_and_author_threads',
'All posts and replies' => 'posts_with_replies',
'Root posts only' => 'posts_no_replies',
'Media only' => 'posts_with_media',
]
],
'include_reposts' => [
'name' => 'Include Reposts?',
'type' => 'checkbox',
'defaultValue' => 'checked'
],
'include_reply_context' => [
'name' => 'Include Reply context?',
'type' => 'checkbox'
],
'verbose_title' => [
'name' => 'Use verbose feed item titles?',
'type' => 'checkbox'
]
]
];
private $profile;
public function getName()
{
if (isset($this->profile)) {
if ($this->profile['handle'] === 'handle.invalid') {
return sprintf('Bluesky - %s', $this->profile['displayName']);
} else {
return sprintf('Bluesky - %s (@%s)', $this->profile['displayName'], $this->profile['handle']);
}
}
return parent::getName();
}
public function getURI()
{
if (isset($this->profile)) {
if ($this->profile['handle'] === 'handle.invalid') {
return self::URI . '/profile/' . $this->profile['did'];
} else {
return self::URI . '/profile/' . $this->profile['handle'];
}
}
return parent::getURI();
}
public function getIcon()
{
if (isset($this->profile)) {
return $this->profile['avatar'];
}
return parent::getIcon();
}
public function getDescription()
{
if (isset($this->profile)) {
return $this->profile['description'];
}
return parent::getDescription();
}
private function parseExternal($external, $did)
{
$description = '';
$externalUri = $external['uri'];
$externalTitle = e($external['title']);
$externalDescription = e($external['description']);
$thumb = $external['thumb'] ?? null;
if (preg_match('/http(|s):\/\/media\.tenor\.com/', $externalUri)) {
//tenor gif embed
$tenorInterstitial = str_replace('media.tenor.com', 'media1.tenor.com/m', $externalUri);
$description .= "<figure><a href=\"$tenorInterstitial\"><img src=\"$externalUri\"/></a><figcaption>$externalTitle</figcaption></figure>";
} else {
//link embed preview
$host = parse_url($externalUri)['host'];
$thumbDesc = $thumb ? ('<img src="https://cdn.bsky.app/img/feed_thumbnail/plain/' . $did . '/' . $thumb['ref']['$link'] . '@jpeg"/>') : '';
$externalDescription = strlen($externalDescription) > 0 ? "<figcaption>($host) $externalDescription</figcaption>" : '';
$description .= '<br><blockquote><b><a href="' . $externalUri . '">' . $externalTitle . '</a></b>';
$description .= '<figure>' . $thumbDesc . $externalDescription . '</figure></blockquote>';
}
return $description;
}
private function textToDescription($record)
{
if (isset($record['value'])) {
$record = $record['value'];
}
$text = $record['text'];
$text_copy = $text;
$text = nl2br(e($text));
if (isset($record['facets'])) {
$facets = $record['facets'];
foreach ($facets as $facet) {
if ($facet['features'][0]['$type'] === 'app.bsky.richtext.facet#link') {
$substring = substr($text_copy, $facet['index']['byteStart'], $facet['index']['byteEnd'] - $facet['index']['byteStart']);
$text = str_replace($substring, '<a href="' . $facet['features'][0]['uri'] . '">' . $substring . '</a>', $text);
}
}
}
return $text;
}
public function collectData()
{
$user_id = $this->getInput('user_id');
$handle_match = preg_match('/(?:[a-zA-Z]*\.)+([a-zA-Z](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)/', $user_id, $handle_res); //gets the TLD in $handle_match[1]
$did_match = preg_match('/did:plc:[a-z2-7]{24}/', $user_id); //https://github.com/did-method-plc/did-method-plc#identifier-syntax
$exclude = ['alt', 'arpa', 'example', 'internal', 'invalid', 'local', 'localhost', 'onion']; //https://en.wikipedia.org/wiki/Top-level_domain#Reserved_domains
if ($handle_match == true && array_search($handle_res[1], $exclude) == false) {
//valid bsky handle
$did = $this->resolveHandle($user_id);
} elseif ($did_match == true) {
//valid DID
$did = $user_id;
} else {
throwClientException('Invalid ATproto handle or DID provided.');
}
$filter = $this->getInput('feed_filter') ?: 'posts_and_author_threads';
$replyContext = $this->getInput('include_reply_context');
$this->profile = $this->getProfile($did);
$authorFeed = $this->getAuthorFeed($did, $filter);
foreach ($authorFeed['feed'] as $post) {
$postRecord = $post['post']['record'];
$item = [];
$item['uri'] = self::URI . '/profile/' . $this->fallbackAuthor($post['post']['author'], 'url') . '/post/' . explode('app.bsky.feed.post/', $post['post']['uri'])[1];
$item['title'] = $this->getInput('verbose_title') ? $this->generateVerboseTitle($post) : strtok($postRecord['text'], "\n");
$item['timestamp'] = strtotime($postRecord['createdAt']);
$item['author'] = $this->fallbackAuthor($post['post']['author'], 'display');
$postAuthorDID = $post['post']['author']['did'];
$postAuthorHandle = $post['post']['author']['handle'] !== 'handle.invalid' ? '<i>@' . $post['post']['author']['handle'] . '</i> ' : '';
$postDisplayName = $post['post']['author']['displayName'] ?? '';
$postDisplayName = e($postDisplayName);
$postUri = $item['uri'];
$url = explode('/', $post['post']['uri']);
$this->logger->debug('https://bsky.app/profile/' . $url[2] . '/post/' . $url[4]);
$description = '';
$description .= '<p>';
//post
$description .= $this->getPostDescription(
$postDisplayName,
$postAuthorHandle,
$postUri,
$postRecord,
'post'
);
if (isset($postRecord['embed']['$type'])) {
//post link embed
if ($postRecord['embed']['$type'] === 'app.bsky.embed.external') {
$description .= $this->parseExternal($postRecord['embed']['external'], $postAuthorDID);
} elseif (
$postRecord['embed']['$type'] === 'app.bsky.embed.recordWithMedia' &&
$postRecord['embed']['media']['$type'] === 'app.bsky.embed.external'
) {
$description .= $this->parseExternal($postRecord['embed']['media']['external'], $postAuthorDID);
}
//post images
if (
$postRecord['embed']['$type'] === 'app.bsky.embed.images' ||
(
$postRecord['embed']['$type'] === 'app.bsky.embed.recordWithMedia' &&
$postRecord['embed']['media']['$type'] === 'app.bsky.embed.images'
)
) {
$images = $post['post']['embed']['images'] ?? $post['post']['embed']['media']['images'];
foreach ($images as $image) {
$description .= $this->getPostImageDescription($image);
}
}
//post video
if (
$postRecord['embed']['$type'] === 'app.bsky.embed.video' ||
(
$postRecord['embed']['$type'] === 'app.bsky.embed.recordWithMedia' &&
$postRecord['embed']['media']['$type'] === 'app.bsky.embed.video'
)
) {
$description .= $this->getPostVideoDescription(
$postRecord['embed']['video'] ?? $postRecord['embed']['media']['video'],
$postAuthorDID
);
}
}
$description .= '</p>';
//quote post
if (
isset($postRecord['embed']) &&
(
$postRecord['embed']['$type'] === 'app.bsky.embed.record' ||
$postRecord['embed']['$type'] === 'app.bsky.embed.recordWithMedia'
) &&
isset($post['post']['embed']['record'])
) {
$description .= '<p>';
$quotedRecord = $post['post']['embed']['record']['record'] ?? $post['post']['embed']['record'];
if (isset($quotedRecord['notFound']) && $quotedRecord['notFound']) { //deleted post
$description .= 'Quoted post deleted.';
} elseif (isset($quotedRecord['detached']) && $quotedRecord['detached']) { //detached quote
$uri_explode = explode('/', $quotedRecord['uri']);
$uri_reconstructed = self::URI . '/profile/' . $uri_explode[2] . '/post/' . $uri_explode[4];
$description .= '<a href="' . $uri_reconstructed . '">Quoted post detached.</a>';
} elseif (isset($quotedRecord['blocked']) && $quotedRecord['blocked']) { //blocked by quote author
$description .= 'Author of quoted post has blocked OP.';
} elseif (
($quotedRecord['$type'] ?? '') === 'app.bsky.feed.defs#generatorView' ||
($quotedRecord['$type'] ?? '') === 'app.bsky.graph.defs#listView'
) {
$description .= $this->getListFeedDescription($quotedRecord);
} elseif (
($quotedRecord['$type'] ?? '') === 'app.bsky.graph.starterpack' ||
($quotedRecord['$type'] ?? '') === 'app.bsky.graph.defs#starterPackViewBasic'
) {
$description .= $this->getStarterPackDescription($post['post']['embed']['record']);
} else {
$quotedAuthorDid = $quotedRecord['author']['did'];
$quotedDisplayName = $quotedRecord['author']['displayName'] ?? '';
$quotedDisplayName = e($quotedDisplayName);
$quotedAuthorHandle = $quotedRecord['author']['handle'] !== 'handle.invalid' ? '<i>@' . $quotedRecord['author']['handle'] . '</i>' : '';
$parts = explode('/', $quotedRecord['uri']);
$quotedPostId = end($parts);
$quotedPostUri = self::URI . '/profile/' . $this->fallbackAuthor($quotedRecord['author'], 'url') . '/post/' . $quotedPostId;
//quoted post - post
$description .= $this->getPostDescription(
$quotedDisplayName,
$quotedAuthorHandle,
$quotedPostUri,
$quotedRecord,
'quote'
);
if (isset($quotedRecord['value']['embed']['$type'])) {
//quoted post - post link embed
if ($quotedRecord['value']['embed']['$type'] === 'app.bsky.embed.external') {
$description .= $this->parseExternal($quotedRecord['value']['embed']['external'], $quotedAuthorDid);
}
//quoted post - post video
if (
$quotedRecord['value']['embed']['$type'] === 'app.bsky.embed.video' ||
(
$quotedRecord['value']['embed']['$type'] === 'app.bsky.embed.recordWithMedia' &&
$quotedRecord['value']['embed']['media']['$type'] === 'app.bsky.embed.video'
)
) {
$description .= $this->getPostVideoDescription(
$quotedRecord['value']['embed']['video'] ?? $quotedRecord['value']['embed']['media']['video'],
$quotedAuthorDid
);
}
//quoted post - post images
if (
$quotedRecord['value']['embed']['$type'] === 'app.bsky.embed.images' ||
(
$quotedRecord['value']['embed']['$type'] === 'app.bsky.embed.recordWithMedia' &&
$quotedRecord['value']['embed']['media']['$type'] === 'app.bsky.embed.images'
)
) {
foreach ($quotedRecord['embeds'] as $embed) {
if (
$embed['$type'] === 'app.bsky.embed.images#view' ||
($embed['$type'] === 'app.bsky.embed.recordWithMedia#view' && $embed['media']['$type'] === 'app.bsky.embed.images#view')
) {
$images = $embed['images'] ?? $embed['media']['images'];
foreach ($images as $image) {
$description .= $this->getPostImageDescription($image);
}
}
}
}
}
}
$description .= '</p>';
}
//reply
if ($replyContext && isset($post['reply']) && isset($post['reply']['parent'])) {
$replyPost = $post['reply']['parent'];
$description .= '<hr/>';
$description .= '<p>';
if (isset($replyPost['notFound']) && $replyPost['notFound']) { //deleted post
$description .= 'Replied to post was deleted.';
} elseif (isset($replyPost['blocked']) && $replyPost['blocked']) { //blocked by quote author
$description .= 'Author of replied to post has blocked OP.';
} else {
$replyPostRecord = $replyPost['record'];
$replyPostAuthorDID = $replyPost['author']['did'];
$replyPostAuthorHandle = $replyPost['author']['handle'] !== 'handle.invalid' ? '<i>@' . $replyPost['author']['handle'] . '</i> ' : '';
$replyPostDisplayName = $replyPost['author']['displayName'] ?? '';
$replyPostDisplayName = e($replyPostDisplayName);
$replyPostUri = self::URI . '/profile/' . $this->fallbackAuthor($replyPost['author'], 'url') . '/post/' . explode('app.bsky.feed.post/', $replyPost['uri'])[1];
// reply post
$description .= $this->getPostDescription(
$replyPostDisplayName,
$replyPostAuthorHandle,
$replyPostUri,
$replyPostRecord,
'reply'
);
if (isset($replyPostRecord['embed']['$type'])) {
//post link embed
if ($replyPostRecord['embed']['$type'] === 'app.bsky.embed.external') {
$description .= $this->parseExternal($replyPostRecord['embed']['external'], $replyPostAuthorDID);
} elseif (
$replyPostRecord['embed']['$type'] === 'app.bsky.embed.recordWithMedia' &&
$replyPostRecord['embed']['media']['$type'] === 'app.bsky.embed.external'
) {
$description .= $this->parseExternal($replyPostRecord['embed']['media']['external'], $replyPostAuthorDID);
}
//post images
if (
$replyPostRecord['embed']['$type'] === 'app.bsky.embed.images' ||
(
$replyPostRecord['embed']['$type'] === 'app.bsky.embed.recordWithMedia' &&
$replyPostRecord['embed']['media']['$type'] === 'app.bsky.embed.images'
)
) {
$images = $replyPost['embed']['images'] ?? $replyPost['embed']['media']['images'];
foreach ($images as $image) {
$description .= $this->getPostImageDescription($image);
}
}
//post video
if (
$replyPostRecord['embed']['$type'] === 'app.bsky.embed.video' ||
(
$replyPostRecord['embed']['$type'] === 'app.bsky.embed.recordWithMedia' &&
$replyPostRecord['embed']['media']['$type'] === 'app.bsky.embed.video'
)
) {
$description .= $this->getPostVideoDescription(
$replyPostRecord['embed']['video'] ?? $replyPostRecord['embed']['media']['video'],
$replyPostAuthorDID
);
}
}
$description .= '</p>';
//quote post
if (
isset($replyPostRecord['embed']) &&
($replyPostRecord['embed']['$type'] === 'app.bsky.embed.record' || $replyPostRecord['embed']['$type'] === 'app.bsky.embed.recordWithMedia') &&
isset($replyPost['embed']['record'])
) {
$description .= '<p>';
$replyQuotedRecord = $replyPost['embed']['record']['record'] ?? $replyPost['embed']['record'];
if (isset($replyQuotedRecord['notFound']) && $replyQuotedRecord['notFound']) { //deleted post
$description .= 'Quoted post deleted.';
} elseif (isset($replyQuotedRecord['detached']) && $replyQuotedRecord['detached']) { //detached quote
$uri_explode = explode('/', $replyQuotedRecord['uri']);
$uri_reconstructed = self::URI . '/profile/' . $uri_explode[2] . '/post/' . $uri_explode[4];
$description .= '<a href="' . $uri_reconstructed . '">Quoted post detached.</a>';
} elseif (isset($replyQuotedRecord['blocked']) && $replyQuotedRecord['blocked']) { //blocked by quote author
$description .= 'Author of quoted post has blocked OP.';
} elseif (
($replyQuotedRecord['$type'] ?? '') === 'app.bsky.feed.defs#generatorView' ||
($replyQuotedRecord['$type'] ?? '') === 'app.bsky.graph.defs#listView'
) {
$description .= $this->getListFeedDescription($replyQuotedRecord);
} elseif (
($replyQuotedRecord['$type'] ?? '') === 'app.bsky.graph.starterpack' ||
($replyQuotedRecord['$type'] ?? '') === 'app.bsky.graph.defs#starterPackViewBasic'
) {
$description .= $this->getStarterPackDescription($replyPost['embed']['record']);
} else {
$quotedAuthorDid = $replyQuotedRecord['author']['did'];
$quotedDisplayName = $replyQuotedRecord['author']['displayName'] ?? '';
$quotedDisplayName = e($quotedDisplayName);
$quotedAuthorHandle = $replyQuotedRecord['author']['handle'] !== 'handle.invalid' ? '<i>@' . $replyQuotedRecord['author']['handle'] . '</i>' : '';
$parts = explode('/', $replyQuotedRecord['uri']);
$quotedPostId = end($parts);
$quotedPostUri = self::URI . '/profile/' . $this->fallbackAuthor($replyQuotedRecord['author'], 'url') . '/post/' . $quotedPostId;
//quoted post - post
$description .= $this->getPostDescription(
$quotedDisplayName,
$quotedAuthorHandle,
$quotedPostUri,
$replyQuotedRecord,
'quote'
);
if (isset($replyQuotedRecord['value']['embed']['$type'])) {
//quoted post - post link embed
if ($replyQuotedRecord['value']['embed']['$type'] === 'app.bsky.embed.external') {
$description .= $this->parseExternal($replyQuotedRecord['value']['embed']['external'], $quotedAuthorDid);
}
//quoted post - post video
if (
$replyQuotedRecord['value']['embed']['$type'] === 'app.bsky.embed.video' ||
(
$replyQuotedRecord['value']['embed']['$type'] === 'app.bsky.embed.recordWithMedia' &&
$replyQuotedRecord['value']['embed']['media']['$type'] === 'app.bsky.embed.video'
)
) {
$description .= $this->getPostVideoDescription(
$replyQuotedRecord['value']['embed']['video'] ?? $replyQuotedRecord['value']['embed']['media']['video'],
$quotedAuthorDid
);
}
//quoted post - post images
if (
$replyQuotedRecord['value']['embed']['$type'] === 'app.bsky.embed.images' ||
(
$replyQuotedRecord['value']['embed']['$type'] === 'app.bsky.embed.recordWithMedia' &&
$replyQuotedRecord['value']['embed']['media']['$type'] === 'app.bsky.embed.images'
)
) {
foreach ($replyQuotedRecord['embeds'] as $embed) {
if (
$embed['$type'] === 'app.bsky.embed.images#view' ||
($embed['$type'] === 'app.bsky.embed.recordWithMedia#view' && $embed['media']['$type'] === 'app.bsky.embed.images#view')
) {
$images = $embed['images'] ?? $embed['media']['images'];
foreach ($images as $image) {
$description .= $this->getPostImageDescription($image);
}
}
}
}
}
}
$description .= '</p>';
}
}
}
$item['content'] = $description;
$this->items[] = $item;
}
}
private function getPostVideoDescription(array $video, $authorDID)
{
//https://video.bsky.app/watch/$did/$cid/thumbnail.jpg
$videoCID = $video['ref']['$link'];
$videoMime = $video['mimeType'];
$thumbnail = "poster=\"https://video.bsky.app/watch/$authorDID/$videoCID/thumbnail.jpg\"" ?? '';
$videoURL = "https://bsky.social/xrpc/com.atproto.sync.getBlob?did=$authorDID&cid=$videoCID";
return "<figure><video loop $thumbnail preload=\"none\" controls src=\"$videoURL\" type=\"$videoMime\"/></figure>";
}
private function getPostImageDescription(array $image)
{
$thumbnailUrl = $image['thumb'];
$fullsizeUrl = $image['fullsize'];
$alt = strlen($image['alt']) > 0 ? '<figcaption>' . e($image['alt']) . '</figcaption>' : '';
return "<figure><a href=\"$fullsizeUrl\"><img src=\"$thumbnailUrl\"></a>$alt</figure>";
}
private function getPostDescription(
string $postDisplayName,
string $postAuthorHandle,
string $postUri,
array $postRecord,
string $type
) {
$description = '';
if ($type === 'quote') {
// Quoted post/reply from bbb @bbb.com:
$postType = isset($postRecord['reply']) ? 'reply' : 'post';
$description .= "<a href=\"$postUri\">Quoted $postType</a> from <b>$postDisplayName</b> $postAuthorHandle:<br>";
} elseif ($type === 'reply') {
// Replying to aaa @aaa.com's post/reply:
$postType = isset($postRecord['reply']) ? 'reply' : 'post';
$description .= "Replying to <b>$postDisplayName</b> $postAuthorHandle's <a href=\"$postUri\">$postType</a>:<br>";
} else {
// aaa @aaa.com posted:
$description .= "<b>$postDisplayName</b> $postAuthorHandle <a href=\"$postUri\">posted</a>:<br>";
}
$description .= $this->textToDescription($postRecord);
return $description;
}
//used if handle verification fails, fallsback to displayName or DID depending on context.
private function fallbackAuthor($author, $reason)
{
if ($author['handle'] === 'handle.invalid') {
switch ($reason) {
case 'url':
return $author['did'];
case 'display':
$displayName = $author['displayName'] ?? '';
return e($displayName);
}
}
return $author['handle'];
}
private function generateVerboseTitle($post)
{
//use "Post by A, replying to B, quoting C" instead of post contents
$title = '';
if (isset($post['reason']) && str_contains($post['reason']['$type'], 'reasonRepost')) {
$title .= 'Repost by ' . $this->fallbackAuthor($post['reason']['by'], 'display') . ', post by ' . $this->fallbackAuthor($post['post']['author'], 'display');
} else {
$title .= 'Post by ' . $this->fallbackAuthor($post['post']['author'], 'display');
}
if (isset($post['reply'])) {
if (isset($post['reply']['parent']['blocked'])) {
$replyAuthor = 'blocked user';
} elseif (isset($post['reply']['parent']['notFound'])) {
$replyAuthor = 'deleted post';
} else {
$replyAuthor = $this->fallbackAuthor($post['reply']['parent']['author'], 'display');
}
$title .= ', replying to ' . $replyAuthor;
}
if (
isset($post['post']['embed']) &&
isset($post['post']['embed']['record']) &&
//if not starter pack, feed or list
($post['post']['embed']['record']['$type'] ?? '') !== 'app.bsky.feed.defs#generatorView' &&
($post['post']['embed']['record']['$type'] ?? '') !== 'app.bsky.graph.defs#listView' &&
($post['post']['embed']['record']['$type'] ?? '') !== 'app.bsky.graph.defs#starterPackViewBasic'
) {
if (isset($post['post']['embed']['record']['blocked'])) {
$quotedAuthor = 'blocked user';
} elseif (isset($post['post']['embed']['record']['notFound'])) {
$quotedAuthor = 'deleted psost';
} elseif (isset($post['post']['embed']['record']['detached'])) {
$quotedAuthor = 'detached post';
} else {
$quotedAuthor = $this->fallbackAuthor($post['post']['embed']['record']['record']['author'] ?? $post['post']['embed']['record']['author'], 'display');
}
$title .= ', quoting ' . $quotedAuthor;
}
return $title;
}
private function resolveHandle($handle)
{
$uri = 'https://public.api.bsky.app/xrpc/com.atproto.identity.resolveHandle?handle=' . urlencode($handle);
$response = json_decode(getContents($uri), true);
return $response['did'];
}
private function getProfile($did)
{
$uri = 'https://public.api.bsky.app/xrpc/app.bsky.actor.getProfile?actor=' . urlencode($did);
$response = json_decode(getContents($uri), true);
return $response;
}
private function getAuthorFeed($did, $filter)
{
$uri = 'https://public.api.bsky.app/xrpc/app.bsky.feed.getAuthorFeed?actor=' . urlencode($did) . '&filter=' . urlencode($filter) . '&limit=30';
$this->logger->debug($uri);
$response = json_decode(getContents($uri), true);
return $response;
}
//Embed for generated feeds and lists
private function getListFeedDescription(array $record): string
{
$feedViewAvatar = isset($record['avatar']) ? '<img src="' . preg_replace('/\/img\/avatar\//', '/img/avatar_thumbnail/', $record['avatar']) . '">' : '';
$feedViewName = e($record['displayName'] ?? $record['name']);
$feedViewDescription = e($record['description'] ?? '');
$authorDisplayName = e($record['creator']['displayName']);
$authorHandle = e($record['creator']['handle']);
$likeCount = isset($record['likeCount']) ? '<br>Liked by ' . e($record['likeCount']) . ' users' : '';
preg_match('/\/([^\/]+)$/', $record['uri'], $matches);
if (($record['purpose'] ?? '') === 'app.bsky.graph.defs#modlist') {
$typeURL = '/lists/';
$typeDesc = 'moderation list';
} elseif (($record['purpose'] ?? '') === 'app.bsky.graph.defs#curatelist') {
$typeURL = '/lists/';
$typeDesc = 'list';
} else {
$typeURL = '/feed/';
$typeDesc = 'feed';
}
$uri = e('https://bsky.app/profile/' . $record['creator']['did'] . $typeURL . $matches[1]);
return <<<END
<blockquote>
<b><a href="{$uri}">{$feedViewName}</a></b><br/>
Bluesky {$typeDesc} by <b>{$authorDisplayName}</b> <i>@{$authorHandle}</i>
<figure>
{$feedViewAvatar}
<figcaption>{$feedViewDescription}{$likeCount}</figcaption>
</figure>
</blockquote>
END;
}
private function getStarterPackDescription(array $record): string
{
if (!isset($record['record'])) {
return 'Failed to get starter pack information.';
}
$starterpackRecord = $record['record'];
$starterpackName = e($starterpackRecord['name']);
$starterpackDescription = e($starterpackRecord['description']);
$creatorDisplayName = e($record['creator']['displayName']);
$creatorHandle = e($record['creator']['handle']);
preg_match('/\/([^\/]+)$/', $starterpackRecord['list'], $matches);
$uri = e('https://bsky.app/starter-pack/' . $record['creator']['did'] . '/' . $matches[1]);
return <<<END
<blockquote>
<b><a href="{$uri}">{$starterpackName}</a></b><br/>
Bluesky starter pack by <b>{$creatorDisplayName}</b> <i>@{$creatorHandle}</i><br/>
{$starterpackDescription}
</blockquote>
END;
}
}

218
bridges/BodaccBridge.php Normal file
View File

@@ -0,0 +1,218 @@
<?php
class BodaccBridge extends BridgeAbstract
{
const NAME = 'BODACC';
const URI = 'https://bodacc-datadila.opendatasoft.com/';
const DESCRIPTION = 'Fetches announces from the French Government "Bulletin Officiel Des Annonces Civiles et Commerciales".';
const CACHE_TIMEOUT = 86400;
const MAINTAINER = 'quent1';
const PARAMETERS = [
'Annonces commerciales' => [
'departement' => [
'name' => 'Département',
'type' => 'list',
'values' => [
'Tous' => null,
'Ain' => '01',
'Aisne' => '02',
'Allier' => '03',
'Alpes-de-Haute-Provence' => '04',
'Hautes-Alpes' => '05',
'Alpes-Maritimes' => '06',
'Ardèche' => '07',
'Ardennes' => '08',
'Ariège' => '09',
'Aube' => '10',
'Aude' => '11',
'Aveyron' => '12',
'Bouches-du-Rhône' => '13',
'Calvados' => '14',
'Cantal' => '15',
'Charente' => '16',
'Charente-Maritime' => '17',
'Cher' => '18',
'Corrèze' => '19',
'Corse-du-Sud' => '2A',
'Haute-Corse' => '2B',
'Côte-d\'Or' => '21',
'Côtes-d\'Armor' => '22',
'Creuse' => '23',
'Dordogne' => '24',
'Doubs' => '25',
'Drôme' => '26',
'Eure' => '27',
'Eure-et-Loir' => '28',
'Finistère' => '29',
'Gard' => '30',
'Haute-Garonne' => '31',
'Gers' => '32',
'Gironde' => '33',
'Hérault' => '34',
'Ille-et-Vilaine' => '35',
'Indre' => '36',
'Indre-et-Loire' => '37',
'Isère' => '38',
'Jura' => '39',
'Landes' => '40',
'Loir-et-Cher' => '41',
'Loire' => '42',
'Haute-Loire' => '43',
'Loire-Atlantique' => '44',
'Loiret' => '45',
'Lot' => '46',
'Lot-et-Garonne' => '47',
'Lozère' => '48',
'Maine-et-Loire' => '49',
'Manche' => '50',
'Marne' => '51',
'Haute-Marne' => '52',
'Mayenne' => '53',
'Meurthe-et-Moselle' => '54',
'Meuse' => '55',
'Morbihan' => '56',
'Moselle' => '57',
'Nièvre' => '58',
'Nord' => '59',
'Oise' => '60',
'Orne' => '61',
'Pas-de-Calais' => '62',
'Puy-de-Dôme' => '63',
'Pyrénées-Atlantiques' => '64',
'Hautes-Pyrénées' => '65',
'Pyrénées-Orientales' => '66',
'Bas-Rhin' => '67',
'Haut-Rhin' => '68',
'Rhône' => '69',
'Haute-Saône' => '70',
'Saône-et-Loire' => '71',
'Sarthe' => '72',
'Savoie' => '73',
'Haute-Savoie' => '74',
'Paris' => '75',
'Seine-Maritime' => '76',
'Seine-et-Marne' => '77',
'Yvelines' => '78',
'Deux-Sèvres' => '79',
'Somme' => '80',
'Tarn' => '81',
'Tarn-et-Garonne' => '82',
'Var' => '83',
'Vaucluse' => '84',
'Vendée' => '85',
'Vienne' => '86',
'Haute-Vienne' => '87',
'Vosges' => '88',
'Yonne' => '89',
'Territoire de Belfort' => '90',
'Essonne' => '91',
'Hauts-de-Seine' => '92',
'Seine-Saint-Denis' => '93',
'Val-de-Marne' => '94',
'Val-d\'Oise' => '95',
'Guadeloupe' => '971',
'Martinique' => '972',
'Guyane' => '973',
'La Réunion' => '974',
'Saint-Pierre-et-Miquelon' => '975',
'Mayotte' => '976',
'Saint-Barthélemy' => '977',
'Saint-Martin' => '978',
'Terres australes et antarctiques françaises' => '984',
'Wallis-et-Futuna' => '986',
'Polynésie française' => '987',
'Nouvelle-Calédonie' => '988',
'Île de Clipperton' => '989'
]
],
'famille' => [
'name' => 'Famille',
'type' => 'list',
'values' => [
'Toutes' => null,
'Annonces diverses' => 'divers',
'Créations' => 'creation',
'Dépôts des comptes' => 'dpc',
'Immatriculations' => 'immatriculation',
'Modifications diverses' => 'modification',
'Procédures collectives' => 'collective',
'Procédures de conciliation' => 'conciliation',
'Procédures de rétablissement professionnel' => 'retablissement_professionnel',
'Radiations' => 'radiation',
'Ventes et cessions' => 'vente'
]
],
'type' => [
'name' => 'Type',
'type' => 'list',
'values' => [
'Tous' => null,
'Avis initial' => 'annonce',
'Avis d\'annulation' => 'annulation',
'Avis rectificatif' => 'rectificatif'
]
]
]
];
public function collectData()
{
$parameters = [
'select' => 'id,dateparution,typeavis_lib,familleavis_lib,commercant,ville,cp',
'order_by' => 'id desc',
'limit' => 50,
];
$where = [];
if (!empty($this->getInput('departement'))) {
$where[] = 'numerodepartement="' . $this->getInput('departement') . '"';
}
if (!empty($this->getInput('famille'))) {
$where[] = 'familleavis="' . $this->getInput('famille') . '"';
}
if (!empty($this->getInput('type'))) {
$where[] = 'typeavis="' . $this->getInput('type') . '"';
}
if ($where !== []) {
$parameters['where'] = implode(' and ', $where);
}
$url = urljoin(self::URI, '/api/explore/v2.1/catalog/datasets/annonces-commerciales/records?' . http_build_query($parameters));
$data = Json::decode(getContents($url), false);
foreach ($data->results as $result) {
if (
!isset(
$result->id,
$result->dateparution,
$result->typeavis_lib,
$result->familleavis_lib,
$result->commercant,
$result->ville,
$result->cp
)
) {
continue;
}
$title = sprintf(
'[%s] %s - %s à %s (%s)',
$result->typeavis_lib,
$result->familleavis_lib,
$result->commercant,
$result->ville,
$result->cp
);
$this->items[] = [
'uid' => $result->id,
'timestamp' => strtotime($result->dateparution),
'title' => $title,
];
}
}
}

1480
bridges/BookMyShowBridge.php Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -1,45 +1,77 @@
<?php
require_once('GelbooruBridge.php');
class BooruprojectBridge extends GelbooruBridge {
class BooruprojectBridge extends DanbooruBridge
{
const MAINTAINER = 'mitsukarenai';
const NAME = 'Booruproject';
const URI = 'https://booru.org/';
const DESCRIPTION = 'Returns images from given page of booruproject';
const PARAMETERS = [
'global' => [
'p' => [
'name' => 'page',
'defaultValue' => 0,
'type' => 'number'
],
't' => [
'name' => 'tags',
'required' => true,
'exampleValue' => 'tagme',
'title' => 'Use "all" to get all posts'
]
],
'Booru subdomain (subdomain.booru.org)' => [
'i' => [
'name' => 'Subdomain',
'required' => true,
'exampleValue' => 'rm'
]
]
];
const MAINTAINER = 'mitsukarenai';
const NAME = 'Booruproject';
const URI = 'http://booru.org/';
const DESCRIPTION = 'Returns images from given page of booruproject';
const PARAMETERS = array(
'global' => array(
'p' => array(
'name' => 'page',
'type' => 'number'
),
't' => array(
'name' => 'tags'
)
),
'Booru subdomain (subdomain.booru.org)' => array(
'i' => array(
'name' => 'Subdomain',
'required' => true
)
)
);
const PATHTODATA = '.thumb';
const IDATTRIBUTE = 'id';
const TAGATTRIBUTE = 'title';
const PIDBYPAGE = 20;
const PIDBYPAGE = 20;
protected function getFullURI()
{
return $this->getURI()
. 'index.php?page=post&s=list&pid='
. ($this->getInput('p') ? ($this->getInput('p') - 1) * static::PIDBYPAGE : '')
. '&tags=' . urlencode($this->getInput('t'));
}
public function getURI(){
if(!is_null($this->getInput('i'))) {
return 'http://' . $this->getInput('i') . '.booru.org/';
}
protected function getTags($element)
{
$tags = parent::getTags($element);
$tags = explode(' ', $tags);
return parent::getURI();
}
// Remove statistics from the tags list (identified by colon)
foreach ($tags as $key => $tag) {
if (strpos($tag, ':') !== false) {
unset($tags[$key]);
}
}
public function getName(){
if(!is_null($this->getInput('i'))) {
return static::NAME . ' ' . $this->getInput('i');
}
return implode(' ', $tags);
}
return parent::getName();
}
public function getURI()
{
if (!is_null($this->getInput('i'))) {
return 'https://' . $this->getInput('i') . '.booru.org/';
}
return parent::getURI();
}
public function getName()
{
if (!is_null($this->getInput('i'))) {
return static::NAME . ' ' . $this->getInput('i');
}
return parent::getName();
}
}

63
bridges/BruegelBridge.php Normal file
View File

@@ -0,0 +1,63 @@
<?php
class BruegelBridge extends BridgeAbstract
{
const NAME = 'Bruegel';
const URI = 'https://www.bruegel.org';
const DESCRIPTION = 'European think-tank commentary and publications.';
const MAINTAINER = 'KappaPrajd';
const PARAMETERS = [
[
'category' => [
'name' => 'Category',
'type' => 'list',
'defaultValue' => '/publications',
'values' => [
'Publications' => '/publications',
'Commentary' => '/commentary'
]
]
]
];
public function getIcon()
{
return self::URI . '/themes/custom/bruegel/assets/favicon/android-icon-72x72.png';
}
public function collectData()
{
$url = self::URI . $this->getInput('category');
$html = getSimpleHTMLDOM($url);
$articles = $html->find('.c-listing__content article');
foreach ($articles as $article) {
$title = $article->find('.c-list-item__title a span', 0)->plaintext;
$content = trim($article->find('.c-list-item__description', 0)->plaintext);
$publishDate = $article->find('.c-list-item__date', 0)->plaintext;
$href = $article->find('.c-list-item__title a', 0)->getAttribute('href');
$item = [
'title' => $title,
'content' => $content,
'timestamp' => strtotime($publishDate),
'uri' => self::URI . $href,
'author' => $this->getAuthor($article),
];
$this->items[] = $item;
}
}
private function getAuthor($article)
{
$authorsElements = $article->find('.c-list-item__authors a');
$authors = array_map(function ($author) {
return $author->plaintext;
}, $authorsElements);
return join(', ', $authors);
}
}

73
bridges/BrutBridge.php Normal file
View File

@@ -0,0 +1,73 @@
<?php
class BrutBridge extends BridgeAbstract
{
const NAME = 'Brut Bridge';
const URI = 'https://www.brut.media';
const DESCRIPTION = 'Returns 10 newest videos by category and edition';
const MAINTAINER = 'VerifiedJoseph';
const PARAMETERS = [[
'category' => [
'name' => 'Category',
'type' => 'list',
'values' => [
'News' => 'news',
'International' => 'international',
'Economy' => 'economy',
'Science and Technology' => 'science-and-technology',
'Entertainment' => 'entertainment',
'Sports' => 'sport',
'Nature' => 'nature',
'Health' => 'health',
],
'defaultValue' => 'news',
],
'edition' => [
'name' => ' Edition',
'type' => 'list',
'values' => [
'United States' => 'us',
'United Kingdom' => 'uk',
'France' => 'fr',
'Spain' => 'es',
'India' => 'in',
'Mexico' => 'mx',
],
'defaultValue' => 'us',
]
]
];
public function collectData()
{
$url = $this->getURI();
$html = getSimpleHTMLDOM($url);
$regex = '/window.__PRELOADED_STATE__ = (.*);/';
preg_match($regex, $html, $parts);
$data = Json::decode($parts[1], false);
foreach ($data->medias->index as $uid => $media) {
$this->items[] = [
'uid' => $uid,
'title' => $media->metadata->slug,
'uri' => $media->share_url,
'timestamp' => $media->published_at,
];
}
}
public function getURI()
{
if (!is_null($this->getInput('edition')) && !is_null($this->getInput('category'))) {
return self::URI . '/' . $this->getInput('edition') . '/' . $this->getInput('category');
}
return parent::getURI();
}
public function getName()
{
if (!is_null($this->getInput('edition')) && !is_null($this->getInput('category'))) {
return $this->getKey('category') . ' - ' . $this->getKey('edition') . ' - Brut.';
}
return parent::getName();
}
}

198
bridges/BugzillaBridge.php Normal file
View File

@@ -0,0 +1,198 @@
<?php
class BugzillaBridge extends BridgeAbstract
{
const NAME = 'Bugzilla Bridge';
const URI = 'https://www.bugzilla.org/';
const DESCRIPTION = 'Bridge for any Bugzilla instance';
const MAINTAINER = 'Yaman Qalieh';
const PARAMETERS = [
'global' => [
'instance' => [
'name' => 'Instance URL',
'required' => true,
'exampleValue' => 'https://bugzilla.mozilla.org'
]
],
'Bug comments' => [
'id' => [
'name' => 'Bug tracking ID',
'type' => 'number',
'required' => true,
'title' => 'Insert bug tracking ID',
'exampleValue' => 121241
],
'limit' => [
'name' => 'Number of comments to return',
'type' => 'number',
'required' => false,
'title' => 'Specify number of comments to return',
'defaultValue' => -1
],
'skiptags' => [
'name' => 'Skip offtopic comments',
'type' => 'checkbox',
'title' => 'Excludes comments tagged as advocacy, metoo, or offtopic from the feed'
]
]
];
const SKIPPED_ACTIVITY = [
'cc' => true,
'comment_tag' => true
];
const SKIPPED_TAGS = ['advocacy', 'metoo', 'offtopic'];
private $instance;
private $bugid;
private $buguri;
private $title;
public function getName()
{
if (!is_null($this->title)) {
return $this->title;
}
return parent::getName();
}
public function getURI()
{
return $this->buguri ?? parent::getURI();
}
public function collectData()
{
$this->instance = rtrim($this->getInput('instance'), '/');
$this->bugid = $this->getInput('id');
$this->buguri = $this->instance . '/show_bug.cgi?id=' . $this->bugid;
$url = $this->instance . '/rest/bug/' . $this->bugid;
$this->getTitle($url);
$this->collectComments($url . '/comment');
$this->collectUpdates($url . '/history');
usort($this->items, function ($a, $b) {
return $b['timestamp'] <=> $a['timestamp'];
});
if ($this->getInput('limit') > 0) {
$this->items = array_slice($this->items, 0, $this->getInput('limit'));
}
}
protected function getTitle($url)
{
// Only request the summary for a faster request
$json = self::getJSON($url . '?include_fields=summary');
$this->title = 'Bug ' . $this->bugid . ' - ' .
$json['bugs'][0]['summary'] . ' - ' .
// Remove https://
substr($this->instance, 8);
}
protected function collectComments($url)
{
$json = self::getJSON($url);
// Array of comments is here
if (!isset($json['bugs'][$this->bugid]['comments'])) {
throwClientException('Cannot find REST endpoint');
}
foreach ($json['bugs'][$this->bugid]['comments'] as $comment) {
$item = [];
if (
$this->getInput('skiptags') and
array_intersect(self::SKIPPED_TAGS, $comment['tags'])
) {
continue;
}
$item['categories'] = $comment['tags'];
$item['uri'] = $this->buguri . '#c' . $comment['count'];
$item['title'] = 'Comment ' . $comment['count'];
$item['timestamp'] = $comment['creation_time'];
$item['author'] = $this->getUser($comment['creator']);
$item['content'] = $comment['text'];
if (isset($comment['is_markdown']) and $comment['is_markdown']) {
$item['content'] = markdownToHtml($item['content']);
}
if (!is_null($comment['attachment_id'])) {
$item['enclosures'] = [$this->instance . '/attachment.cgi?id=' . $comment['attachment_id']];
}
$this->items[] = $item;
}
}
protected function collectUpdates($url)
{
$json = self::getJSON($url);
// Array of changesets which contain an array of changes
if (!isset($json['bugs']['0']['history'])) {
throwClientException('Cannot find REST endpoint');
}
foreach ($json['bugs']['0']['history'] as $changeset) {
$author = $this->getUser($changeset['who']);
$timestamp = $changeset['when'];
foreach ($changeset['changes'] as $change) {
// Skip updates to the cc list and comment tagging
if (isset(self::SKIPPED_ACTIVITY[$change['field_name']])) {
continue;
}
$item = [];
$item['uri'] = $this->buguri;
$item['title'] = 'Updated';
$item['timestamp'] = $timestamp;
$item['author'] = $author;
$item['content'] = ucfirst($change['field_name']) . ': ' .
($change['removed'] === '' ? '[nothing]' : $change['removed']) . ' -> ' .
($change['added'] === '' ? '[nothing]' : $change['added']);
$this->items[] = $item;
}
}
}
protected function getUser($user)
{
// Check if the user endpoint is available
if ($this->loadCacheValue($this->instance . 'userEndpointClosed')) {
return $user;
}
$cache = $this->loadCacheValue($this->instance . $user);
if ($cache) {
return $cache;
}
$url = $this->instance . '/rest/user/' . $user . '?include_fields=real_name';
try {
$json = self::getJSON($url);
if (isset($json['error']) and $json['error']) {
throw new Exception();
}
} catch (Exception $e) {
$this->saveCacheValue($this->instance . 'userEndpointClosed', true);
return $user;
}
$username = $json['users']['0']['real_name'];
if (empty($username)) {
$username = $user;
}
$this->saveCacheValue($this->instance . $user, $username);
return $username;
}
protected static function getJSON($url)
{
$headers = [
'Accept: application/json',
];
return json_decode(getContents($url, $headers), true);
}
}

220
bridges/BukowskisBridge.php Normal file
View File

@@ -0,0 +1,220 @@
<?php
class BukowskisBridge extends BridgeAbstract
{
const NAME = 'Bukowskis';
const URI = 'https://www.bukowskis.com';
const DESCRIPTION = 'Fetches info about auction objects from Bukowskis auction house';
const MAINTAINER = 'Qluxzz';
const PARAMETERS = [[
'category' => [
'name' => 'Category',
'type' => 'list',
'values' => [
'All categories' => '',
'Art' => [
'All' => 'art',
'Classic Art' => 'art.classic-art',
'Classic Finnish Art' => 'art.classic-finnish-art',
'Classic Swedish Art' => 'art.classic-swedish-art',
'Contemporary' => 'art.contemporary',
'Modern Finnish Art' => 'art.modern-finnish-art',
'Modern International Art' => 'art.modern-international-art',
'Modern Swedish Art' => 'art.modern-swedish-art',
'Old Masters' => 'art.old-masters',
'Other' => 'art.other',
'Photographs' => 'art.photographs',
'Prints' => 'art.prints',
'Sculpture' => 'art.sculpture',
'Swedish Old Masters' => 'art.swedish-old-masters',
],
'Asian Ceramics & Works of Art' => [
'All' => 'asian-ceramics-works-of-art',
'Other' => 'asian-ceramics-works-of-art.other',
'Porcelain' => 'asian-ceramics-works-of-art.porcelain',
],
'Books & Manuscripts' => [
'All' => 'books-manuscripts',
'Books' => 'books-manuscripts.books',
],
'Carpets, rugs & textiles' => [
'All' => 'carpets-rugs-textiles',
'European' => 'carpets-rugs-textiles.european',
'Oriental' => 'carpets-rugs-textiles.oriental',
'Rest of the world' => 'carpets-rugs-textiles.rest-of-the-world',
'Scandinavian' => 'carpets-rugs-textiles.scandinavian',
],
'Ceramics & porcelain' => [
'All' => 'ceramics-porcelain',
'Ceramic ware' => 'ceramics-porcelain.ceramic-ware',
'European' => 'ceramics-porcelain.european',
'Rest of the world' => 'ceramics-porcelain.rest-of-the-world',
'Scandinavian' => 'ceramics-porcelain.scandinavian',
],
'Collectibles' => [
'All' => 'collectibles',
'Advertising & Retail' => 'collectibles.advertising-retail',
'Memorabilia' => 'collectibles.memorabilia',
'Movies & music' => 'collectibles.movies-music',
'Other' => 'collectibles.other',
'Retro & Popular Culture' => 'collectibles.retro-popular-culture',
'Technica & Nautica' => 'collectibles.technica-nautica',
'Toys' => 'collectibles.toys',
],
'Design' => [
'All' => 'design',
'Art glass' => 'design.art-glass',
'Furniture' => 'design.furniture',
'Other' => 'design.other',
],
'Folk art' => [
'All' => 'folk-art',
'All categories' => 'lots',
],
'Furniture' => [
'All' => 'furniture',
'Armchairs & Sofas' => 'furniture.armchairs-sofas',
'Cabinets & Bureaus' => 'furniture.cabinets-bureaus',
'Chairs' => 'furniture.chairs',
'Garden furniture' => 'furniture.garden-furniture',
'Mirrors' => 'furniture.mirrors',
'Other' => 'furniture.other',
'Shelves & Book cases' => 'furniture.shelves-book-cases',
'Tables' => 'furniture.tables',
],
'Glassware' => [
'All' => 'glassware',
'Glassware' => 'glassware.glassware',
'Other' => 'glassware.other',
],
'Jewellery' => [
'All' => 'jewellery',
'Bracelets' => 'jewellery.bracelets',
'Brooches' => 'jewellery.brooches',
'Earrings' => 'jewellery.earrings',
'Necklaces & Pendants' => 'jewellery.necklaces-pendants',
'Other' => 'jewellery.other',
'Rings' => 'jewellery.rings',
],
'Lighting' => [
'All' => 'lighting',
'Candle sticks & Candelabras' => 'lighting.candle-sticks-candelabras',
'Ceiling lights' => 'lighting.ceiling-lights',
'Chandeliers' => 'lighting.chandeliers',
'Floor lights' => 'lighting.floor-lights',
'Other' => 'lighting.other',
'Table lights' => 'lighting.table-lights',
'Wall lights' => 'lighting.wall-lights',
],
'Militaria' => [
'All' => 'militaria',
'Honors & Medals' => 'militaria.honors-medals',
'Other militaria' => 'militaria.other-militaria',
'Weaponry' => 'militaria.weaponry',
],
'Miscellaneous' => [
'All' => 'miscellaneous',
'Brass, Copper & Pewter' => 'miscellaneous.brass-copper-pewter',
'Nickel silver' => 'miscellaneous.nickel-silver',
'Oriental' => 'miscellaneous.oriental',
'Other' => 'miscellaneous.other',
],
'Silver' => [
'All' => 'silver',
'Candle sticks' => 'silver.candle-sticks',
'Cups & Bowls' => 'silver.cups-bowls',
'Cutlery' => 'silver.cutlery',
'Other' => 'silver.other',
],
'Timepieces' => [
'All' => 'timepieces',
'Other' => 'timepieces.other',
'Pocket watches' => 'timepieces.pocket-watches',
'Table clocks' => 'timepieces.table-clocks',
'Wrist watches' => 'timepieces.wrist-watches',
],
'Vintage & Fashion' => [
'All' => 'vintage-fashion',
'Accessories' => 'vintage-fashion.accessories',
'Bags & Trunks' => 'vintage-fashion.bags-trunks',
'Clothes' => 'vintage-fashion.clothes',
],
]
],
'sort_order' => [
'name' => 'Sort order',
'type' => 'list',
'values' => [
'Ending soon' => 'ending',
'Most recent' => 'recent',
'Most bids' => 'most',
'Fewest bids' => 'fewest',
'Lowest price' => 'lowest',
'Highest price' => 'highest',
'Lowest estimate' => 'low',
'Highest estimate' => 'high',
'Alphabetical' => 'alphabetical',
],
],
'language' => [
'name' => 'Language',
'type' => 'list',
'values' => [
'English' => 'en',
'Swedish' => 'sv',
'Finnish' => 'fi'
],
],
]];
const CACHE_TIMEOUT = 3600; // 1 hour
private $title;
public function collectData()
{
$baseUrl = 'https://www.bukowskis.com';
$category = $this->getInput('category');
$language = $this->getInput('language');
$sort_order = $this->getInput('sort_order');
$url = $baseUrl . '/' . $language . '/lots';
if ($category) {
$url = $url . '/category/' . $category;
}
if ($sort_order) {
$url = $url . '/sort/' . $sort_order;
}
$html = getSimpleHTMLDOM($url);
$this->title = htmlspecialchars_decode($html->find('title', 0)->innertext);
foreach ($html->find('div.c-lot-index-lot') as $lot) {
$title = $lot->find('a.c-lot-index-lot__title', 0)->plaintext;
$relative_url = $lot->find('a.c-lot-index-lot__link', 0)->href;
$images = json_decode(
htmlspecialchars_decode(
$lot
->find('img.o-aspect-ratio__image', 0)
->getAttribute('data-thumbnails')
)
);
$this->items[] = [
'title' => $title,
'uri' => $baseUrl . $relative_url,
'uid' => $relative_url,
'content' => count($images) > 0 ? "<img src='$images[0]'/><br/>$title" : $title,
'enclosures' => array_slice($images, 1),
];
}
}
public function getName()
{
return $this->title ?: parent::getName();
}
}

View File

@@ -0,0 +1,90 @@
<?php
class BundesbankBridge extends BridgeAbstract
{
const PARAM_LANG = 'lang';
const LANG_EN = 'en';
const LANG_DE = 'de';
const NAME = 'Bundesbank Bridge';
const URI = 'https://www.bundesbank.de/';
const DESCRIPTION = 'Returns the latest studies of the Bundesbank (Germany)';
const MAINTAINER = 'logmanoriginal';
const CACHE_TIMEOUT = 86400; // 24 hours
const PARAMETERS = [
[
self::PARAM_LANG => [
'name' => 'Language',
'type' => 'list',
'defaultValue' => self::LANG_DE,
'values' => [
'English' => self::LANG_EN,
'Deutsch' => self::LANG_DE
]
]
]
];
public function getIcon()
{
return self::URI . 'resource/crblob/1890/a7f48ee0ae35348748121770ba3ca009/mL/favicon-ico-data.ico';
}
public function getURI()
{
switch ($this->getInput(self::PARAM_LANG)) {
case self::LANG_EN:
return self::URI . 'en/publications/reports/studies';
case self::LANG_DE:
return self::URI . 'de/publikationen/berichte/studien';
}
return parent::getURI();
}
public function collectData()
{
$html = getSimpleHTMLDOM($this->getURI());
$html = defaultLinkTo($html, $this->getURI());
foreach ($html->find('ul.resultlist li') as $study) {
$item = [];
$item['uri'] = $study->find('.teasable__link', 0)->href;
// Get title without child elements (i.e. subtitle)
$title = $study->find('.teasable__title div.h2', 0);
foreach ($title->children as &$child) {
$child->outertext = '';
}
$item['title'] = $title->innertext;
// Add subtitle to the content if it exists
$item['content'] = '';
if ($subtitle = $study->find('.teasable__subtitle', 0)) {
$item['content'] .= '<strong>' . $study->find('.teasable__subtitle', 0)->plaintext . '</strong>';
}
$teasable = $study->find('.teasable__text', 0);
$teasableText = $teasable->plaintext ?? '';
$item['content'] .= '<p>' . $teasableText . '</p>';
$item['timestamp'] = strtotime($study->find('.teasable__date', 0)->plaintext);
// Downloads and older studies don't have images
if ($study->find('.teasable__image', 0)) {
$item['enclosures'] = [
$study->find('.teasable__image img', 0)->src
];
}
$this->items[] = $item;
}
}
}

View File

@@ -0,0 +1,92 @@
<?php
class BundestagParteispendenBridge extends BridgeAbstract
{
const MAINTAINER = 'mibe';
const NAME = 'Deutscher Bundestag - Parteispenden';
const URI = 'https://www.bundestag.de/parlament/praesidium/parteienfinanzierung/fundstellen50000';
const CACHE_TIMEOUT = 86400; // 24h
const DESCRIPTION = 'Returns the latest "soft money" donations to parties represented in the German Bundestag.';
const CONTENT_TEMPLATE = <<<TMPL
<p><b>Partei:</b><br>%s</p>
<p><b>Spendenbetrag:</b><br>%s</p>
<p><b>Spender:</b><br>%s</p>
<p><b>Eingang der Spende:</b><br>%s</p>
TMPL;
public function getIcon()
{
return 'https://www.bundestag.de/static/appdata/includes/images/layout/favicon.ico';
}
public function collectData()
{
$ajaxUri = <<<URI
https://www.bundestag.de/ajax/filterlist/de/parlament/praesidium/parteienfinanzierung/fundstellen50000/462002-462002
URI;
// Get the main page
$html = getSimpleHTMLDOMCached($ajaxUri, self::CACHE_TIMEOUT);
// Build the URL from the first anchor element. The list is sorted by year, descending, so the first element is the current year.
$firstAnchor = $html->find('a', 0)
or throwServerException('Could not find the proper HTML element.');
$url = $firstAnchor->href;
// Get the actual page with the soft money donations
$html = getSimpleHTMLDOMCached($url, self::CACHE_TIMEOUT);
$rows = $html->find('table.table > tbody > tr')
or throwServerException('Could not find the proper HTML elements.');
foreach ($rows as $row) {
$item = $this->generateItemFromRow($row);
if (is_array($item)) {
$item['uri'] = $url;
$this->items[] = $item;
}
}
}
private function generateItemFromRow(simple_html_dom_node $row)
{
// The row must have 5 columns. There are monthly header rows, which are ignored here.
if (count($row->children) != 5) {
return null;
}
$item = [];
// | column | paragraph inside column
$party = $row->children[0]->children[0]->innertext;
$amount = $row->children[1]->children[0]->innertext . ' €';
$donor = $row->children[2]->children[0]->innertext;
$date = $row->children[3]->children[0]->innertext;
$dip = $row->children[4]->children[0]->find('a.dipLink', 0);
// Strip whitespace from date string.
$date = str_replace(' ', '', $date);
$content = sprintf(self::CONTENT_TEMPLATE, $party, $amount, $donor, $date);
$item = [
'title' => $party . ': ' . $amount,
'content' => $content,
'uid' => sha1($content),
];
// Try to get the link to the official document
if ($dip != null) {
$item['enclosures'] = [$dip->href];
}
// Try to parse the date
$dateTime = DateTime::createFromFormat('d.m.Y', $date);
if ($dateTime !== false) {
$item['timestamp'] = $dateTime->getTimestamp();
}
return $item;
}
}

View File

@@ -0,0 +1,28 @@
<?php
class BundesverbandFuerFreieKammernBridge extends XPathAbstract
{
const NAME = 'Bundesverband für freie Kammern e.V.';
const URI = 'https://www.bffk.de/aktuelles/aktuelle-nachrichten.html';
const DESCRIPTION = 'Aktuelle Nachrichten';
const MAINTAINER = 'hleskien';
const FEED_SOURCE_URL = 'https://www.bffk.de/aktuelles/aktuelle-nachrichten.html';
//const XPATH_EXPRESSION_FEED_ICON = './/link[@rel="icon"]/@href';
const XPATH_EXPRESSION_ITEM = '//ul[@class="article-list"]/li';
const XPATH_EXPRESSION_ITEM_TITLE = './/a/text()';
const XPATH_EXPRESSION_ITEM_CONTENT = './/a/text()';
const XPATH_EXPRESSION_ITEM_URI = './/a/@href';
//const XPATH_EXPRESSION_ITEM_AUTHOR = './/';
const XPATH_EXPRESSION_ITEM_TIMESTAMP = './/span/i';
//const XPATH_EXPRESSION_ITEM_ENCLOSURES = './';
//const XPATH_EXPRESSION_ITEM_CATEGORIES = './/';
protected function formatItemTimestamp($value)
{
$value = trim($value, '()');
$dti = DateTimeImmutable::createFromFormat('d.m.Y', $value);
$dti = $dti->setTime(0, 0, 0);
return $dti->getTimestamp();
}
}

Some files were not shown because too many files have changed in this diff Show More