1
0
mirror of https://github.com/RSS-Bridge/rss-bridge.git synced 2025-08-25 17:31:25 +02:00

Compare commits

..

63 Commits

Author SHA1 Message Date
Florent V.
096e398e41 [SamMobileBridge] New brige to fetches the latest security patches for Samsung devices (#4676)
* [SamMobileBridge] Fetches the latest security patches for Samsung devices

* [SamMobileBridge] Add date handling

* [SamMobileBridge] Remove empty spaces

* [SamMobileBridge] add strict_types

---------

Co-authored-by: Florent VIOLLEAU <florent.violleau@samsic.fr>
2025-08-20 16:30:22 +02:00
Florent V.
b423b13bd5 [EdfPricesBrige] Update for dom change (#4675)
Co-authored-by: Florent VIOLLEAU <florent.violleau@samsic.fr>
2025-08-19 18:23:27 +02:00
Mynacol
e30698f12f [GolemBridge] Add multi-page headings
On multi-page articles like [1], some paragraph headers were missing
because they are headers of the article pages.

These headers were previously removed in
c5f586497f for being redundant with the
original header. The article at [1] proves us wrong, but I added a logic
to ignore truly duplicate headers.

[1] https://www.golem.de/news/es-muss-nicht-immer-apple-sein-fuenf-ueberzeugende-airpods-pro-alternativen-im-test-2508-195000.html
2025-08-17 14:56:42 +02:00
Lukas Nabakowski
876d3c8ae7 [ZDFMediathekBridge] add bridge (#4672)
* Add ZDFMediathekBridge

* Declare strict types = 1
2025-08-15 16:46:32 +02:00
Matt DeMoss
ee4f85cc94 pcgamer: meta tag change (#4670)
* pcgamer: the parsely tags are gone, use different tags

* apply phpcs.xml rules
2025-08-14 19:06:39 +02:00
tillcash
1b584b4551 [CybernewsBridge] add bridge (#4665)
* [CybernewsBridge] add bridge

* [CybernewsBridge] fix lint

* [CybernewsBridge] add header

* [CybernewsBridge] fix url

* [CybernewsBridge] fix url 2

* [CybernewsBridge] revert header

* [CybernewsBridge] refactor

* [CybernewsBridge] final

* [CybernewsBridge] lint
2025-08-14 14:31:47 +02:00
xnand-dot-xyz
3a9e398228 [ModrinthBridge] Add bridge (#4651)
* [ModrinthBridge] Add bridge

Support for querying updates to projects on https://modrinth.com

May need modification,  and I'm alright with the maintainer name being changed or cleared if actual maintenance is expected

* Added declare and fixed linting errors

* Skip parsing lists if null, and trim trailing space
2025-08-14 14:28:33 +02:00
Simone Dotto
2e387eb9d6 [SubitoBridge] Add bridge (#1800) (#4628)
* [SubitoBridge] Add bridge (issue #1800)

* php 74 compat

* user-agent blocking bypass

* constant variable access

* strict types

---------

Co-authored-by: Simone Dotto <simonedotto@proton.me>
2025-08-14 14:26:16 +02:00
User123698745
9b6fa7cd97 [prtester] improve prtester.py and prhtmlgenerator.yml for running in forks (#4313)
* [prtester] support forks to upload to their own "rss-bridge-tests"

add parameter "--artifact-base-url" and "--artifact-directory"

* [prtester] review feedback: add 'github.event.number' fallback to 'none'
2025-08-14 08:17:42 +02:00
Mynacol
5382dee516 [GolemBridge] Fix removal of affiliate images
On
https://www.golem.de/news/anlage-in-etfs-was-alternativen-zum-msci-world-bringen-2508-199041.html
the affiliate box isn't properly filtered out.
The reason seems to be switching from a `div` to an `aside` element.
HTML source fragment:
```html
<aside class="gbox_affiliate" data-nosnippet>
    <div class="gbox_attribution"></div>
    <div class="gbox_fx1">
<a href="https://www.financeads.net/tc.php?t=36731C67231788T" target="_blank" rel="nofollow" onclick="_gcpx.push(['ev','d','rklmbox/14387']); return true;"><img src="https://scr3.golem.de/screenshots/affiliate/14
387/9caaa476f979dcf7457395f39ac9ed9f.png" alt=""></a>
        <div class="gbox_fx2">
            <div class="gbox_title">Tagesgeld, Festgeld, ETFs, Aktien und mehr bei raisin</div>
            <div><a class="gbox_btn" data-cta="Jetzt Investmentm&ouml;glichkeiten bei raisin entdecken" href="https://www.financeads.net/tc.php?t=36731C67231788T" target="_blank" rel="nofollow" onclick="_gcpx.push(['ev','d','rklmbox/14387']); return true;"></a></div>
        </div>
    </div>
<!-- /gbox --></aside>
```
2025-08-13 12:15:26 +02:00
Mynacol
b60556ffb4 [HeiseBridge] Remove "Videos by heise" ads
This seems to be a new middle-of-content self-ad.
Seen on https://heise.de/-10519045

The code snippet in that case was:
```html
<div class="ad ad--inread">

  <div class="ad--inread-header">
    <p class="ad--inread-header__text">
      Videos by heise
    </p>

    <div class="ad--inread-header__more">
      <button class="ad--inread-header-menu-toggle" popovertarget="ad--inread-header-menu">
        mehr Videos
        <svg fill="none" height="24" viewbox="0 0 24 24" width="24" xmlns="http://www.w3.org/2000/svg">
          <path d="M8.625 12.0023C8.625 12.2094 8.45711 12.3773 8.25 12.3773C8.04289 12.3773 7.875 12.2094 7.875 12.0023C7.875 11.7952 8.04289 11.6273 8.25 11.6273C8.45711 11.6273 8.625 11.7952 8.625 12.0023ZM8.625 12.0023H8.25M12.375 12.0023C12.375 12.2094 12.2071 12.3773 12 12.3773C11.7929 12.3773 11.625 12.2094 11.625 12.0023C11.625 11.7952 11.7929 11.6273 12 11.6273C12.2071 11.6273 12.375 11.7952 12.375 12.0023ZM12.375 12.0023H12M16.125 12.0023C16.125 12.2094 15.9571 12.3773 15.75 12.3773C15.5429 12.3773 15.375 12.2094 15.375 12.0023C15.375 11.7952 15.5429 11.6273 15.75 11.6273C15.9571 11.6273 16.125 11.7952 16.125 12.0023ZM16.125 12.0023H15.75M21 12.0023C21 16.9729 16.9706 21.0023 12 21.0023C7.02944 21.0023 3 16.9729 3 12.0023C3 7.03176 7.02944 3.00232 12 3.00232C16.9706 3.00232 21 7.03176 21 12.0023Z" stroke="#777" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5"></path>
        </svg>
      </button>

      <div class="ad--inread-header-menu" id="ad--inread-header-menu" popover>
        <ul class="a-u-mb-0">
          <li>
            <a class="ad--inread-header-menu-link" href="https://www.youtube.com/@ct3003" target="_blank">
              c&#39;t 3003
            </a>
          </li>
          <li>
            <a class="ad--inread-header-menu-link" href="https://www.youtube.com/heiseonline" target="_blank">
              heise &amp; ct
            </a>
          </li>
          <li>
            <a class="ad--inread-header-menu-link" href="https://peertube.heise.de/" target="_blank">
              Peertube
            </a>
          </li>
        </ul>
      </div>
    </div>
  </div>

  <figure class="video video--fullwidth">
    <a-video entry-id="25969" height="9" instant is-target-video-playlist style="aspect-ratio: 16 / 9" type="targetvideo" width="16"></a-video>
  </figure>
</div>
```

Hence filtering anything with the class `ad` or `ad--inread` gets rid of
it.
2025-08-12 21:25:42 +02:00
Dag
37174f01e5 fix: throw client exception in some bridges (#4661) 2025-08-08 02:24:13 +02:00
Dag
a599f4ba83 fix: dont log user errors (#4660) 2025-08-08 02:16:43 +02:00
Dag
81ce9c9483 fix: introduce system env var, remove debug mode (#4658)
* fix: introduce system env var

* docs

* docs
2025-08-08 01:38:12 +02:00
Dag
a128c05a97 docs: emphasize strict types (#4657) 2025-08-05 21:06:40 +02:00
Dag
9caa043fe1 lint: add returnClientError and returnServerError to forbiddenFcuntions (#4656) 2025-08-05 20:55:04 +02:00
Dag
f11571ae78 refactor: rename functions (#4655)
returnClientError => throwClientException
returnServerError => throwServerException

New convenience function: throwRateLimitException

Old functions are kept but deprecated.
2025-08-05 20:44:40 +02:00
Dag
b39964cee3 chore: prepare for aug 2025 release (#4654) 2025-08-05 19:50:27 +02:00
Joseph
9c43921a33 [FirstLookMediaTechBridge] Remove bridge (#4653)
Website no longer exists
2025-08-04 22:57:35 +02:00
Joseph
9e2975048f [AskfmBridge] Remove bridge (#4652)
Website closed in December 2024 https://web.archive.org/web/20241129120541/https://about.ask.fm/closure-notice-the-platform-to-be-deactivated-december-1-2024/
2025-08-04 22:56:27 +02:00
Joseph
fb153f9a92 [DansTonChatBridge] Remove bridge (#4650)
bridge is broken and website has native feeds.

https://danstonchat.com/category/quote/feed
2025-08-04 17:19:24 +02:00
Joseph
20fec74c63 [DailymotionBridge] Fetch playlist title from API (#4649) 2025-08-04 15:41:04 +02:00
Simone Dotto
b5f90f8d47 [AmazonPriceTracker] Fix price not shown, new default source (#4631)
Fixes issue #4586

Co-authored-by: Simone Dotto <simonedotto@proton.me>
2025-08-04 14:31:43 +02:00
shaun
aba38845d2 [YoutubeCommunityTabsBridge] Rename Community→Posts to fix broken bridge (#4606)
* youtube community posts are just called "Posts" now

* finish renaming Community -> Posts

* add feedName fallbacks (thanks @Mar-Koeh)

* rename YouTubePostsTabBridge back to YouTubeCommunityTabBridge

* fix linter error by breaking up long expression

* fix optional-chaining regression by using ‘?? null’
2025-08-04 14:30:48 +02:00
Joseph
1211ac63d9 Update DailymotionBridge.php (#4648) 2025-08-04 14:28:16 +02:00
Joseph
640503168e [FirefoxAddonsBridge] Minor change to item content html (#4647) 2025-08-04 14:27:40 +02:00
Arnav Jain
93de253d01 [GoComicsBridge] cache individual comic page for 24h (#4646) 2025-08-04 14:27:19 +02:00
User123698745
6ec4da854f [FallGuysBridge] fix: handle new data structure (#4640)
* [FallGuysBridge] fix: handle new data structure

* [FallGuysBridge] review feedback: removed mixed
2025-08-04 01:36:44 +02:00
Dag
e5f9fe6251 lint (#4645) 2025-08-04 01:36:15 +02:00
Dag
47c9983e16 fix: dont cache basic auth response (#4644) 2025-08-04 01:32:36 +02:00
Sandro
69eda522c8 Mention php extension filter (#4608)
While trying around to minimize my installation, I noticed that this
extension is nowhere mentioned.
2025-08-04 01:09:38 +02:00
User123698745
172e7eb280 [prtester] fix wrong pr check fail when refactoring code (the bridge html output has not changed) (#4642)
ignore "nothing to commit, working tree clean"
2025-08-04 01:08:25 +02:00
User123698745
acb9373c10 [DRKBlutspendeBridge] add offers to content & add caption to images & use cached request (#4641) 2025-08-04 01:07:41 +02:00
Joseph
85497238c5 Update HaveIBeenPwnedBridge.php (#4638) 2025-08-04 00:58:09 +02:00
Marcin Morawski
a2334838a6 Fix deprecations (#4636)
* Fix PHP 8.4 deprecation

Implicitly marking parameter as nullable is deprecated, the explicit nullable type must be used instead

* [github workflow] Add additional php versions
2025-08-04 00:55:50 +02:00
mruac
c65fbd5543 [BlueskyBridge] Fix cases for missing reply post context and QoL fix for video loading (#4635)
* added fix for missing reply post context

* qol fix - no preload on videos
2025-08-04 00:50:12 +02:00
sysadminstory
e241f3dcde [PepperBridgeAbstract, DealabsBridge, HotUKDealsBridge, MydealsBridge] Adapt RSS bridge to website content update; remove country of origin due to missing data (#4634)
Website use now "vue3" and some class and attributes have changed their
names : bridge was updated to use the new class and attribute names

Country of origin has been removed from the deal list : it's for now
disabled, but code is still present in the bridge, in case the website
enable it again.
2025-08-04 00:48:27 +02:00
Pavel Korytov
16bb6156a5 [UniverseTodayBridge] Add bridge (#4627) 2025-08-04 00:22:50 +02:00
Pavel Korytov
9f8dc411a4 [InstituteForTheStudyOfWarBridge] Increase caching time (#4626) 2025-08-04 00:21:57 +02:00
July
5b97899734 [FanaticalBridge] Create a new bridge (#4624)
Provides a fairly barebones bridge for Fanatical bundles:
- Tags detail bundle tiers and prices
- Contents name and link to each bundle item
- Images for each item are in enclosures
2025-08-04 00:21:04 +02:00
July
8ae2c2e3c3 [HumbleBundleBridge] Overhaul to include more information (#4621)
* [HumbleBundleBridge] Overhaul to include more information

* [HumbleBundleBridge] Remove use of named args in calls

PHP 7.4 lacks named arg support and fails unit tests
2025-08-04 00:20:00 +02:00
July
9ec6ae39a2 [ComickBridge] Add new bridge (#4625)
Makes new brige for manga from comick.io. Like the CubariProxyBridge,
can provide manga page images in feed entry content or enclosures.
2025-08-04 00:19:08 +02:00
July
3517cda4a5 [YouTubeFeedExpanderBridge] More reliable channel icons (#4622) 2025-08-04 00:17:30 +02:00
July
52be29d3ec [AnnasArchiveBridge] Fix book list CSS selector (#4619) 2025-08-04 00:17:01 +02:00
July
696aed22cc [CubariProxyBridge] Replace MangaSee with WeebCentral (#4618) 2025-08-04 00:16:30 +02:00
July
e394be7ca5 [KemonoBridge] Add search query support (#4620) 2025-08-04 00:16:14 +02:00
jaydeethree
3835f290c1 Update GOGBridge to use GOG's REST API. I have tested this locally and it seems to work correctly. (#4616) 2025-08-04 00:14:51 +02:00
Nomis
c7de5c95be Update 06_Public_Hosts.md (#4614)
Remove bridge.easter.fr
2025-08-04 00:12:38 +02:00
Tobias Alexander Franke
71808aaa81 [WarhammerComBridge] Bridge for Warhammer Community blog (#4610)
* [WarhammerComBridge] Bridge for Warhammer Community blog

* Fix Linter issues
2025-08-04 00:10:58 +02:00
Anton Smirnov
2ca696c1cf [EpicGamesFreeBridge] productSlug can be null; also add a universal future-proof-ish fallback (#4595)
* productSlug can be null, do more discovery, add fallback

* productSlug can be garbage too, remove it completely
2025-08-03 23:59:42 +02:00
Sebastian K
c90b98b965 Error handling in ExplosmBridge (#4600)
Skip further processing if element was not found to avoid errors
2025-08-03 23:58:24 +02:00
Quentin B.
8e880de3d2 [CentreFranceBridge] Fix parser following website update (#4596)
* [CentreFranceBridge] Fix parser following website update

* [CentreFranceBridge] Fix empty content

* [CentreFranceBridge] Fix title parsing
2025-08-03 23:52:06 +02:00
Tone
bfa6c4c080 [HeiseBridge] removes language-info-text, add archive.is link for people without subscription (#4594)
* [HeiseBridge] removes language-info-text, add archive.is link for people without subscription

* fix annoying phpcs
2025-08-03 23:50:54 +02:00
User.
5ab938ada7 [WaggaCouncilBridge] Add bridge (#4593)
Co-authored-by: Scrub000 <scrub@example.com>
2025-08-03 23:49:10 +02:00
Petr Prenghy
4d2fe2f12d [NasestrechaBridge] Add bridge (#4591)
* Add files via upload

Bridge for NaseStrecha.cz - NaseStrecha.cz is a specialized Czech news and advice portal focusing on roofs, construction, and home improvement, offering reliable expert guidance on roofing materials, insulation, and energy-saving techniques nasestrecha.cz . It is run by the team behind the Strechy-Solar-Remeslo trade fair and includes up-to-date news, practical tips, and industry events

* phpcs fix

* Bridge for i4wifi.cz for product news.
The website i4wifi.cz is a wholesale distributor specializing in wireless, networking, and photovoltaic equipment, offering products from brands like MikroTik, Ubiquiti, and Hikvision. It provides a wide range of network solutions, technical support, and training services for businesses and professional installers in the Czech Republic and beyond.
2025-08-03 23:46:35 +02:00
Mynacol
4c0b97d605 [ZeitBridge] Add advertorial marker to article
So users are aware that it's a paid article.

Some might still find them interesting, so we cannot just filter them
away.
2025-07-20 01:35:28 +02:00
Mynacol
1d5bcba41f [ZeitBridge] Hide magazine ads in articles
Test article: https://www.zeit.de/campus/2025/03/kyoto-university-abschlussfeier-kostueme-japan
2025-07-20 01:35:28 +02:00
Mynacol
d19ce75d4b Merge pull request #4613 from Mynacol/golem-add-table
[GolemBridge] Add tables to content
2025-07-16 13:53:53 +02:00
Mynacol
bfbe2abdce [GolemBridge] Add tables to content
For example the following article has such tables that should be
included:
https://www.golem.de/news/immobilien-mieten-oder-kaufen-warum-es-dabei-nicht-nur-ums-geld-geht-2507-197406.html
2025-07-16 11:50:00 +00:00
Jonathan Kay
354cea09a7 [GoComicsBridge] Add fallback when link to current comic is missing (#4589) 2025-06-08 21:57:41 +02:00
sysadminstory
8dada08e69 [IdealoBridge] Bypass bot protection (#4588)
Add some headers (User-Agent, Accept, Accept-Language) and activate
compression to bypass the bot protection
2025-06-07 23:31:02 +02:00
Jonathan Kay
514b3edf0b [GoComicsBridge] Fix for JSON being removed (#4585)
- Now redirects to first comic from landing page
- Switched to meta tags
2025-06-05 23:41:20 +02:00
Tobias Alexander Franke
7aa54602cf [FabBridge] Pull 100% discounted items via Fab API (#4584)
* [FabBridge] Pull 100% discounted items via Fab API

* [FabBridge] Linter fixes
2025-06-04 22:15:28 +02:00
157 changed files with 2732 additions and 1020 deletions

35
.github/prtester.py vendored
View File

@@ -21,13 +21,10 @@ class Instance:
name = ''
url = ''
def main(instances: Iterable[Instance], with_upload: bool, with_reduced_upload: bool, title: str, output_file: str):
def main(instances: Iterable[Instance], with_artifacts: bool, with_reduced_artifacts: bool, artifacts_directory: str, artifacts_base_url: str, title: str, output_file: str):
start_date = datetime.now()
prid = os.getenv('PR')
artifact_base_url = f'https://rss-bridge.github.io/rss-bridge-tests/prs/{prid}'
artifact_directory = os.getcwd()
for file in glob.glob(f'*{ARTIFACT_FILE_EXTENSION}', root_dir=artifact_directory):
for file in glob.glob(f'*{ARTIFACT_FILE_EXTENSION}', root_dir=artifacts_directory):
os.remove(file)
table_rows = []
@@ -38,10 +35,10 @@ def main(instances: Iterable[Instance], with_upload: bool, with_reduced_upload:
table_rows += testBridges(
instance=instance,
bridge_cards=bridge_cards,
with_upload=with_upload,
with_reduced_upload=with_reduced_upload,
artifact_directory=artifact_directory,
artifact_base_url=artifact_base_url) # run the main scraping code with the list of bridges
with_artifacts=with_artifacts,
with_reduced_artifacts=with_reduced_artifacts,
artifacts_directory=artifacts_directory,
artifacts_base_url=artifacts_base_url) # run the main scraping code with the list of bridges
with open(file=output_file, mode='w+', encoding='utf-8') as file:
table_rows_value = '\n'.join(sorted(table_rows))
file.write(f'''
@@ -53,7 +50,7 @@ def main(instances: Iterable[Instance], with_upload: bool, with_reduced_upload:
*last change: {start_date.strftime("%A %Y-%m-%d %H:%M:%S")}*
'''.strip())
def testBridges(instance: Instance, bridge_cards: Iterable, with_upload: bool, with_reduced_upload: bool, artifact_directory: str, artifact_base_url: str) -> Iterable:
def testBridges(instance: Instance, bridge_cards: Iterable, with_artifacts: bool, with_reduced_artifacts: bool, artifacts_directory: str, artifacts_base_url: str) -> Iterable:
instance_suffix = ''
if instance.name:
instance_suffix = f' ({instance.name})'
@@ -155,12 +152,12 @@ def testBridges(instance: Instance, bridge_cards: Iterable, with_upload: bool, w
status_is_ok = status == '';
if status_is_ok:
status = '✔️'
if with_upload and (not with_reduced_upload or not status_is_ok):
if with_artifacts and (not with_reduced_artifacts or not status_is_ok):
filename = f'{bridge_name} {form_number}{instance_suffix}{ARTIFACT_FILE_EXTENSION}'
filename = re.sub(r'[^a-z0-9 \_\-\.]', '', filename, flags=re.I).replace(' ', '_')
with open(file=f'{artifact_directory}/{filename}', mode='wb') as file:
with open(file=f'{artifacts_directory}/{filename}', mode='wb') as file:
file.write(page_text)
artifact_url = f'{artifact_base_url}/{filename}'
artifact_url = f'{artifacts_base_url}/{filename}'
table_rows.append(f'| {bridge_name} | [{form_number} {context_name}{instance_suffix}]({artifact_url}) | {status} |')
form_number += 1
return table_rows
@@ -177,8 +174,10 @@ def getFirstLine(value: str) -> str:
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--instances', nargs='+')
parser.add_argument('--no-upload', action='store_true')
parser.add_argument('--reduced-upload', action='store_true')
parser.add_argument('--no-artifacts', action='store_true')
parser.add_argument('--reduced-artifacts', action='store_true')
parser.add_argument('--artifacts-directory', default=os.getcwd())
parser.add_argument('--artifacts-base-url', default='')
parser.add_argument('--title', default='Pull request artifacts')
parser.add_argument('--output-file', default=os.getcwd() + '/comment.txt')
args = parser.parse_args()
@@ -201,8 +200,10 @@ if __name__ == '__main__':
instances.append(instance)
main(
instances=instances,
with_upload=not args.no_upload,
with_reduced_upload=args.reduced_upload and not args.no_upload,
with_artifacts=not args.no_artifacts,
with_reduced_artifacts=args.reduced_artifacts and not args.no_artifacts,
artifacts_directory=args.artifacts_directory,
artifacts_base_url=args.artifacts_base_url,
title=args.title,
output_file=args.output_file
);

View File

@@ -5,24 +5,29 @@ on:
branches: [ master ]
jobs:
check-bridges:
checks:
name: Check if bridges were changed
runs-on: ubuntu-latest
outputs:
BRIDGES: ${{ steps.check1.outputs.BRIDGES }}
BRIDGES: ${{ steps.check_bridges.outputs.BRIDGES }}
WITH_UPLOAD: ${{ steps.check_upload.outputs.WITH_UPLOAD }}
steps:
- name: Check number of bridges
id: check1
id: check_bridges
run: |
PR=${{github.event.number}};
PR=${{ github.event.number || 'none' }};
wget https://patch-diff.githubusercontent.com/raw/$GITHUB_REPOSITORY/pull/$PR.patch;
bridgeamount=$(cat $PR.patch | grep "\bbridges/[A-Za-z0-9]*Bridge\.php\b" | sed "s=.*\bbridges/\([A-Za-z0-9]*\)Bridge\.php\b.*=\1=g" | sort | uniq | wc -l);
echo "BRIDGES=$bridgeamount" >> "$GITHUB_OUTPUT"
- name: "Check upload token secret RSSTESTER_ACTION is set"
id: check_upload
run: |
echo "WITH_UPLOAD=$([ -n "${{ secrets.RSSTESTER_ACTION }}" ] && echo "true" || echo "false")" >> "$GITHUB_OUTPUT"
test-pr:
name: Generate HTML
runs-on: ubuntu-latest
needs: check-bridges
if: needs.check-bridges.outputs.BRIDGES > 0
needs: checks
if: needs.checks.outputs.BRIDGES > 0
env:
PYTHONUNBUFFERED: 1
# Needs additional permissions https://github.com/actions/first-interaction/issues/10#issuecomment-1041402989
@@ -34,7 +39,7 @@ jobs:
repository: ${{github.event.pull_request.head.repo.full_name}}
- name: Check out rss-bridge
run: |
PR=${{github.event.number}};
PR=${{ github.event.number || 'none' }};
wget -O requirements.txt https://raw.githubusercontent.com/$GITHUB_REPOSITORY/${{ github.event.pull_request.base.ref }}/.github/prtester-requirements.txt;
wget https://raw.githubusercontent.com/$GITHUB_REPOSITORY/${{ github.event.pull_request.base.ref }}/.github/prtester.py;
wget https://patch-diff.githubusercontent.com/raw/$GITHUB_REPOSITORY/pull/$PR.patch;
@@ -60,14 +65,12 @@ jobs:
id: testrun
run: |
mkdir results;
python prtester.py;
python prtester.py --artifacts-base-url "https://${{ github.repository_owner }}.github.io/${{ vars.ARTIFACTS_REPO || 'rss-bridge-tests' }}/prs/${{ github.event.number || 'none' }}";
body="$(cat comment.txt)";
body="${body//'%'/'%25'}";
body="${body//$'\n'/'%0A'}";
body="${body//$'\r'/'%0D'}";
echo "bodylength=${#body}" >> $GITHUB_OUTPUT
env:
PR: ${{ github.event.number }}
- name: Upload generated tests
uses: actions/upload-artifact@v4
id: upload-generated-tests
@@ -94,33 +97,31 @@ jobs:
name: Upload tests
runs-on: ubuntu-latest
needs: test-pr
if: needs.checks.outputs.WITH_UPLOAD == 'true'
steps:
- uses: actions/checkout@v4
with:
repository: 'RSS-Bridge/rss-bridge-tests'
repository: "${{ github.repository_owner }}/${{ vars.ARTIFACTS_REPO || 'rss-bridge-tests' }}"
ref: 'main'
token: ${{ secrets.RSSTESTER_ACTION }}
- name: Setup git config
run: |
git config --global user.name "GitHub Actions"
git config --global user.email "<>"
- name: Download tests
uses: actions/download-artifact@v4
with:
name: tests
- name: Move tests
run: |
cd prs
mkdir -p ${{github.event.number}}
cd ${{github.event.number}}
DIRECTORY="$GITHUB_WORKSPACE/prs/${{ github.event.number || 'none' }}"
rm -rf $DIRECTORY
mkdir -p $DIRECTORY
cd $DIRECTORY
mv -f $GITHUB_WORKSPACE/*.html .
- name: Commit and push generated tests
run: |
export COMMIT_MESSAGE="Added tests for PR ${{github.event.number}}"
export COMMIT_MESSAGE="Added tests for PR ${{ github.event.number || 'none' }}"
git add .
git commit -m "$COMMIT_MESSAGE"
git commit -m "$COMMIT_MESSAGE" || exit 0
git push

View File

@@ -11,7 +11,7 @@ jobs:
runs-on: ubuntu-22.04
strategy:
matrix:
php-versions: ['7.4', '8.0', '8.1']
php-versions: ['7.4', '8.0', '8.1', '8.2', '8.3', '8.4']
steps:
- uses: actions/checkout@v4
- uses: shivammathur/setup-php@v2

View File

@@ -46,7 +46,7 @@ Requires minimum PHP 7.4.
* `TwitchBridge`: [Fetches videos from channel](https://rss-bridge.org/bridge01/#bridge-TwitchBridge)
* `XPathBridge`: [Scrape out a feed using XPath expressions](https://rss-bridge.org/bridge01/#bridge-XPathBridge)
* `YoutubeBridge`: [Fetches videos by username/channel/playlist/search](https://rss-bridge.org/bridge01/#bridge-YoutubeBridge)
* `YouTubeCommunityTabBridge`: [Fetches posts from a channel's community tab](https://rss-bridge.org/bridge01/#bridge-YouTubeCommunityTabBridge)
* `YouTubeCommunityTabBridge`: [Fetches posts from a channel's Posts tab](https://rss-bridge.org/bridge01/#bridge-YouTubeCommunityTabBridge)
## Tutorial
@@ -321,13 +321,23 @@ The sqlite files (db, wal and shm) are not writeable.
rm cache/*
### How to create a new bridge from scratch
### How to create a completely new bridge
New code files MUST have `declare(strict_types=1);` at the top of file:
```php
<?php
declare(strict_types=1);
```
Create the new bridge in e.g. `bridges/BearBlogBridge.php`:
```php
<?php
declare(strict_types=1);
class BearBlogBridge extends BridgeAbstract
{
const NAME = 'BearBlog (bearblog.dev)';
@@ -359,14 +369,6 @@ enabled_bridges[] = TwitchBridge
enabled_bridges[] = GettrBridge
```
### How to enable debug mode
The
[debug mode](https://rss-bridge.github.io/rss-bridge/For_Developers/Debug_mode.html)
disables the majority of caching operations.
enable_debug_mode = true
### How to switch to memcached as cache backend
```

View File

@@ -22,8 +22,8 @@ class ConnectivityAction implements ActionInterface
public function __invoke(Request $request): Response
{
if (!Debug::isEnabled()) {
return new Response('This action is only available in debug mode!', 403);
if (Configuration::getConfig('system', 'env') !== 'dev') {
return new Response('This action is only available in dev environment!', 403);
}
$bridgeName = $request->get('bridge');

View File

@@ -89,12 +89,12 @@ class DisplayAction implements ActionInterface
$bridge->collectData();
$items = $bridge->getItems();
} catch (\Throwable $e) {
if ($e instanceof RateLimitException) {
// These are internally generated by bridges
$this->logger->info(sprintf('RateLimitException in DisplayAction(%s): %s', $bridge->getShortName(), create_sane_exception_message($e)));
if ($e instanceof ClientException) {
$this->logger->debug(sprintf('Exception in DisplayAction(%s): %s', $bridge->getShortName(), create_sane_exception_message($e)));
} elseif ($e instanceof RateLimitException) {
$this->logger->debug(sprintf('Exception in DisplayAction(%s): %s', $bridge->getShortName(), create_sane_exception_message($e)));
return new Response(render(__DIR__ . '/../templates/exception.html.php', ['e' => $e]), 429);
}
if ($e instanceof HttpException) {
} elseif ($e instanceof HttpException) {
if (in_array($e->getCode(), [429, 503])) {
// Log with debug, immediately reproduce and return
$this->logger->debug(sprintf('Exception in DisplayAction(%s): %s', $bridge->getShortName(), create_sane_exception_message($e)));
@@ -102,7 +102,6 @@ class DisplayAction implements ActionInterface
}
// Some other status code which we let fail normally (but don't log it)
} else {
// Log error if it's not an HttpException
$this->logger->error(sprintf('Exception in DisplayAction(%s)', $bridge->getShortName()), ['e' => $e]);
}
$errorOutput = Configuration::getConfig('error', 'output');

View File

@@ -71,7 +71,7 @@ class ARDAudiothekBridge extends BridgeAbstract
$pathComponents = explode('/', $path);
if (empty($pathComponents)) {
returnClientError('Path may not be empty');
throwClientException('Path may not be empty');
}
if (count($pathComponents) < 2) {
$showID = $pathComponents[0];

View File

@@ -65,7 +65,7 @@ class ARDMediathekBridge extends BridgeAbstract
$pathComponents = explode('/', $this->getInput('path'));
if (empty($pathComponents)) {
returnClientError('Path may not be empty');
throwClientException('Path may not be empty');
}
if (count($pathComponents) < 2) {
$showID = $pathComponents[0];

View File

@@ -57,7 +57,7 @@ class AllocineFRBridge extends BridgeAbstract
if (array_key_exists($category, $categories)) {
return static::URI . $this->getLastSeasonURI($categories[$category]);
} else {
returnClientError('Emission inconnue');
throwClientException('Emission inconnue');
}
}

View File

@@ -2,7 +2,7 @@
class AmazonPriceTrackerBridge extends BridgeAbstract
{
const MAINTAINER = 'captn3m0, sal0max';
const MAINTAINER = 'captn3m0, sal0max, bagnacauda';
const NAME = 'Amazon Price Tracker';
const URI = 'https://www.amazon.com/';
const CACHE_TIMEOUT = 3600; // 1h
@@ -13,7 +13,7 @@ class AmazonPriceTrackerBridge extends BridgeAbstract
'asin' => [
'name' => 'ASIN',
'required' => true,
'exampleValue' => 'B071GB1VMQ',
'exampleValue' => 'B0923XT6K7',
// https://stackoverflow.com/a/12827734
'pattern' => 'B[\dA-Z]{9}|\d{9}(X|\d)',
],
@@ -169,19 +169,23 @@ EOT;
private function scrapePriceTwister($html)
{
$str = $html->find('.twister-plus-buying-options-price-data', 0);
$json = $html->find('.twister-plus-buying-options-price-data', 0);
if ($json == null) {
return null;
}
$data = json_decode($str->innertext, true);
if (count($data) === 1) {
$data = $data[0];
$data = json_decode($json->innertext, true);
foreach ($data as $key => $value) {
$value = $value[0];
return [
'displayPrice' => $data['displayPrice'],
'currency' => $data['currency'],
'shipping' => '0',
'displayPrice' => $value['displayPrice'],
'price' => $value['priceAmount'],
'currency' => $value['currencySymbol'],
'shipping' => null,
];
}
return false;
return null;
}
private function scrapePriceGeneric($html)
@@ -206,9 +210,21 @@ EOT;
}
$priceString = str_replace(str_split(self::WHITESPACE), '', $priceDiv->plaintext);
preg_match('/(\d+\.\d{0,2})/', $priceString, $matches);
$price = null;
$priceFound = false;
// find longest repeated string
for ($offset = 0; $offset < strlen($priceString); $offset++) {
for ($length = 1; substr_count($priceString, substr($priceString, $offset, $length + 1)) >= 2; $length++) {
$priceFound = true;
}
if ($priceFound) {
$price = substr($priceString, $offset, $length);
break;
}
}
$price = $matches[0] ?? null;
$currency = str_replace($price, '', $priceString);
if ($price != null && $currency != null) {
@@ -216,7 +232,7 @@ EOT;
'price' => $price,
'displayPrice' => null,
'currency' => $currency,
'shipping' => '0'
'shipping' => null
];
}
return $default;
@@ -227,7 +243,7 @@ EOT;
$html = $this->getHtml();
$this->title = $this->getTitle($html);
$image = $this->getImage($html);
$data = $this->scrapePriceGeneric($html);
$data = $this->scrapePriceTwister($html) ?? $this->scrapePriceGeneric($html);
// render
$content = '';
@@ -236,7 +252,7 @@ EOT;
$price = sprintf('%s %s', $data['price'], $data['currency']);
}
$content .= sprintf('%s<br>Price: %s', $image, $price);
if ($data['shipping'] !== '0') {
if ($data['shipping'] !== null) {
$content .= sprintf('<br>Shipping: %s %s</br>', $data['shipping'], $data['currency']);
}

View File

@@ -152,7 +152,7 @@ class AnidexBridge extends BridgeAbstract
}
}
if (empty($results) && empty($this->getInput('q'))) {
returnServerError('No results from Anidex: ' . $search_url);
throwServerException('No results from Anidex: ' . $search_url);
}
//Process each item individually

View File

@@ -126,7 +126,7 @@ class AnnasArchiveBridge extends BridgeAbstract
return;
}
$elements = $list->find('.w-full > .mb-4 > div');
$elements = $list->find('#aarecord-list > div');
foreach ($elements as $element) {
// stop added entries once partial match list starts
if (str_contains($element->innertext, 'partial match')) {

View File

@@ -71,7 +71,7 @@ class AppleMusicBridge extends BridgeAbstract
$result = $json->results;
if (!is_array($result) || count($result) == 0) {
returnServerError('There is no artist with id "' . $this->getInput('artist') . '".');
throwServerException('There is no artist with id "' . $this->getInput('artist') . '".');
}
return $result;

View File

@@ -1,80 +0,0 @@
<?php
class AskfmBridge extends BridgeAbstract
{
const MAINTAINER = 'az5he6ch, logmanoriginal';
const NAME = 'Ask.fm Answers';
const URI = 'https://ask.fm/';
const CACHE_TIMEOUT = 300; //5 min
const DESCRIPTION = 'Returns answers from an Ask.fm user';
const PARAMETERS = [
'Ask.fm username' => [
'u' => [
'name' => 'Username',
'required' => true,
'exampleValue' => 'ApprovedAndReal'
]
]
];
public function collectData()
{
$html = getSimpleHTMLDOM($this->getURI());
$html = defaultLinkTo($html, self::URI);
foreach ($html->find('article.streamItem-answer') as $element) {
$item = [];
$item['uri'] = $element->find('a.streamItem_meta', 0)->href;
$question = trim($element->find('header.streamItem_header', 0)->innertext);
$item['title'] = trim(
htmlspecialchars_decode(
$element->find('header.streamItem_header', 0)->plaintext,
ENT_QUOTES
)
);
$item['timestamp'] = strtotime($element->find('time', 0)->datetime);
$var = $element->find('div.streamItem_content', 0);
$answer = trim($var->innertext ?? '');
// This probably should be cleaned up, especially for YouTube embeds
if ($visual = $element->find('div.streamItem_visual', 0)) {
$visual = $visual->innertext;
}
// Fix tracking links, also doesn't work
foreach ($element->find('a') as $link) {
if (strpos($link->href, 'l.ask.fm') !== false) {
$link->href = $link->plaintext;
}
}
$item['content'] = '<p>' . $question
. '</p><p>' . $answer
. '</p><p>' . $visual . '</p>';
$this->items[] = $item;
}
}
public function getName()
{
if (!is_null($this->getInput('u'))) {
return self::NAME . ' : ' . $this->getInput('u');
}
return parent::getName();
}
public function getURI()
{
if (!is_null($this->getInput('u'))) {
return self::URI . urlencode($this->getInput('u'));
}
return parent::getURI();
}
}

View File

@@ -66,10 +66,10 @@ class AssociatedPressNewsBridge extends BridgeAbstract
{
switch ($this->getInput('topic')) {
case 'Podcasts':
returnClientError('Podcasts topic feed is not supported');
throwClientException('Podcasts topic feed is not supported');
break;
case 'PressReleases':
returnClientError('PressReleases topic feed is not supported');
throwClientException('PressReleases topic feed is not supported');
break;
default:
$this->collectCardData();
@@ -110,7 +110,7 @@ class AssociatedPressNewsBridge extends BridgeAbstract
$tagContents = json_decode($json, true);
if (empty($tagContents['tagObjs'])) {
returnClientError('Topic not found: ' . $this->getInput('topic'));
throwClientException('Topic not found: ' . $this->getInput('topic'));
}
$this->feedName = $tagContents['tagObjs'][0]['name'];

View File

@@ -94,7 +94,7 @@ class BakaUpdatesMangaReleasesBridge extends BridgeAbstract
// content is an unstructured pile of divs, ugly to parse
$cols = $html->find('div#main_content div.row > div.text');
if (!$cols) {
returnServerError('No releases');
throwServerException('No releases');
}
$rows = array_slice(

View File

@@ -123,7 +123,7 @@ class BandcampBridge extends BridgeAbstract
$json = json_decode($content);
if ($json->ok !== true) {
returnServerError('Invalid response');
throwServerException('Invalid response');
}
foreach ($json->items as $entry) {
@@ -165,7 +165,7 @@ class BandcampBridge extends BridgeAbstract
$regex = '/band_id=(\d+)/';
if (preg_match($regex, $html, $matches) == false) {
returnServerError('Unable to find band ID on: ' . $this->getURI());
throwServerException('Unable to find band ID on: ' . $this->getURI());
}
$band_id = $matches[1];
@@ -196,7 +196,7 @@ class BandcampBridge extends BridgeAbstract
case 'By album':
$regex = '/album=(\d+)/';
if (preg_match($regex, $html, $matches) == false) {
returnServerError('Unable to find album ID on: ' . $this->getURI());
throwServerException('Unable to find album ID on: ' . $this->getURI());
}
$album_id = $matches[1];

View File

@@ -154,7 +154,7 @@ class BlueskyBridge extends BridgeAbstract
//valid DID
$did = $user_id;
} else {
returnClientError('Invalid ATproto handle or DID provided.');
throwClientException('Invalid ATproto handle or DID provided.');
}
$filter = $this->getInput('feed_filter') ?: 'posts_and_author_threads';
@@ -178,10 +178,8 @@ class BlueskyBridge extends BridgeAbstract
$postDisplayName = e($postDisplayName);
$postUri = $item['uri'];
if (Debug::isEnabled()) {
$url = explode('/', $post['post']['uri']);
$this->logger->debug('https://bsky.app/profile/' . $url[2] . '/post/' . $url[4]);
}
$url = explode('/', $post['post']['uri']);
$this->logger->debug('https://bsky.app/profile/' . $url[2] . '/post/' . $url[4]);
$description = '';
$description .= '<p>';
@@ -330,157 +328,163 @@ class BlueskyBridge extends BridgeAbstract
}
//reply
if ($replyContext && isset($post['reply']) && !isset($post['reply']['parent']['notFound'])) {
if ($replyContext && isset($post['reply']) && isset($post['reply']['parent'])) {
$replyPost = $post['reply']['parent'];
$replyPostRecord = $replyPost['record'];
$description .= '<hr/>';
$description .= '<p>';
$replyPostAuthorDID = $replyPost['author']['did'];
$replyPostAuthorHandle = $replyPost['author']['handle'] !== 'handle.invalid' ? '<i>@' . $replyPost['author']['handle'] . '</i> ' : '';
$replyPostDisplayName = $replyPost['author']['displayName'] ?? '';
$replyPostDisplayName = e($replyPostDisplayName);
$replyPostUri = self::URI . '/profile/' . $this->fallbackAuthor($replyPost['author'], 'url') . '/post/' . explode('app.bsky.feed.post/', $replyPost['uri'])[1];
if (isset($replyPost['notFound']) && $replyPost['notFound']) { //deleted post
$description .= 'Replied to post was deleted.';
} elseif (isset($replyPost['blocked']) && $replyPost['blocked']) { //blocked by quote author
$description .= 'Author of replied to post has blocked OP.';
} else {
$replyPostRecord = $replyPost['record'];
$replyPostAuthorDID = $replyPost['author']['did'];
$replyPostAuthorHandle = $replyPost['author']['handle'] !== 'handle.invalid' ? '<i>@' . $replyPost['author']['handle'] . '</i> ' : '';
$replyPostDisplayName = $replyPost['author']['displayName'] ?? '';
$replyPostDisplayName = e($replyPostDisplayName);
$replyPostUri = self::URI . '/profile/' . $this->fallbackAuthor($replyPost['author'], 'url') . '/post/' . explode('app.bsky.feed.post/', $replyPost['uri'])[1];
// reply post
$description .= $this->getPostDescription(
$replyPostDisplayName,
$replyPostAuthorHandle,
$replyPostUri,
$replyPostRecord,
'reply'
);
// reply post
$description .= $this->getPostDescription(
$replyPostDisplayName,
$replyPostAuthorHandle,
$replyPostUri,
$replyPostRecord,
'reply'
);
if (isset($replyPostRecord['embed']['$type'])) {
//post link embed
if ($replyPostRecord['embed']['$type'] === 'app.bsky.embed.external') {
$description .= $this->parseExternal($replyPostRecord['embed']['external'], $replyPostAuthorDID);
} elseif (
$replyPostRecord['embed']['$type'] === 'app.bsky.embed.recordWithMedia' &&
$replyPostRecord['embed']['media']['$type'] === 'app.bsky.embed.external'
) {
$description .= $this->parseExternal($replyPostRecord['embed']['media']['external'], $replyPostAuthorDID);
}
//post images
if (
$replyPostRecord['embed']['$type'] === 'app.bsky.embed.images' ||
(
if (isset($replyPostRecord['embed']['$type'])) {
//post link embed
if ($replyPostRecord['embed']['$type'] === 'app.bsky.embed.external') {
$description .= $this->parseExternal($replyPostRecord['embed']['external'], $replyPostAuthorDID);
} elseif (
$replyPostRecord['embed']['$type'] === 'app.bsky.embed.recordWithMedia' &&
$replyPostRecord['embed']['media']['$type'] === 'app.bsky.embed.images'
)
) {
$images = $replyPost['embed']['images'] ?? $replyPost['embed']['media']['images'];
foreach ($images as $image) {
$description .= $this->getPostImageDescription($image);
$replyPostRecord['embed']['media']['$type'] === 'app.bsky.embed.external'
) {
$description .= $this->parseExternal($replyPostRecord['embed']['media']['external'], $replyPostAuthorDID);
}
//post images
if (
$replyPostRecord['embed']['$type'] === 'app.bsky.embed.images' ||
(
$replyPostRecord['embed']['$type'] === 'app.bsky.embed.recordWithMedia' &&
$replyPostRecord['embed']['media']['$type'] === 'app.bsky.embed.images'
)
) {
$images = $replyPost['embed']['images'] ?? $replyPost['embed']['media']['images'];
foreach ($images as $image) {
$description .= $this->getPostImageDescription($image);
}
}
//post video
if (
$replyPostRecord['embed']['$type'] === 'app.bsky.embed.video' ||
(
$replyPostRecord['embed']['$type'] === 'app.bsky.embed.recordWithMedia' &&
$replyPostRecord['embed']['media']['$type'] === 'app.bsky.embed.video'
)
) {
$description .= $this->getPostVideoDescription(
$replyPostRecord['embed']['video'] ?? $replyPostRecord['embed']['media']['video'],
$replyPostAuthorDID
);
}
}
$description .= '</p>';
//post video
//quote post
if (
$replyPostRecord['embed']['$type'] === 'app.bsky.embed.video' ||
(
$replyPostRecord['embed']['$type'] === 'app.bsky.embed.recordWithMedia' &&
$replyPostRecord['embed']['media']['$type'] === 'app.bsky.embed.video'
)
isset($replyPostRecord['embed']) &&
($replyPostRecord['embed']['$type'] === 'app.bsky.embed.record' || $replyPostRecord['embed']['$type'] === 'app.bsky.embed.recordWithMedia') &&
isset($replyPost['embed']['record'])
) {
$description .= $this->getPostVideoDescription(
$replyPostRecord['embed']['video'] ?? $replyPostRecord['embed']['media']['video'],
$replyPostAuthorDID
);
}
}
$description .= '</p>';
$description .= '<p>';
$replyQuotedRecord = $replyPost['embed']['record']['record'] ?? $replyPost['embed']['record'];
//quote post
if (
isset($replyPostRecord['embed']) &&
($replyPostRecord['embed']['$type'] === 'app.bsky.embed.record' || $replyPostRecord['embed']['$type'] === 'app.bsky.embed.recordWithMedia') &&
isset($replyPost['embed']['record'])
) {
$description .= '<p>';
$replyQuotedRecord = $replyPost['embed']['record']['record'] ?? $replyPost['embed']['record'];
if (isset($replyQuotedRecord['notFound']) && $replyQuotedRecord['notFound']) { //deleted post
$description .= 'Quoted post deleted.';
} elseif (isset($replyQuotedRecord['detached']) && $replyQuotedRecord['detached']) { //detached quote
$uri_explode = explode('/', $replyQuotedRecord['uri']);
$uri_reconstructed = self::URI . '/profile/' . $uri_explode[2] . '/post/' . $uri_explode[4];
$description .= '<a href="' . $uri_reconstructed . '">Quoted post detached.</a>';
} elseif (isset($replyQuotedRecord['blocked']) && $replyQuotedRecord['blocked']) { //blocked by quote author
$description .= 'Author of quoted post has blocked OP.';
} elseif (
($replyQuotedRecord['$type'] ?? '') === 'app.bsky.feed.defs#generatorView' ||
($replyQuotedRecord['$type'] ?? '') === 'app.bsky.graph.defs#listView'
) {
$description .= $this->getListFeedDescription($replyQuotedRecord);
} elseif (
($replyQuotedRecord['$type'] ?? '') === 'app.bsky.graph.starterpack' ||
($replyQuotedRecord['$type'] ?? '') === 'app.bsky.graph.defs#starterPackViewBasic'
) {
$description .= $this->getStarterPackDescription($replyPost['embed']['record']);
} else {
$quotedAuthorDid = $replyQuotedRecord['author']['did'];
$quotedDisplayName = $replyQuotedRecord['author']['displayName'] ?? '';
$quotedDisplayName = e($quotedDisplayName);
$quotedAuthorHandle = $replyQuotedRecord['author']['handle'] !== 'handle.invalid' ? '<i>@' . $replyQuotedRecord['author']['handle'] . '</i>' : '';
if (isset($replyQuotedRecord['notFound']) && $replyQuotedRecord['notFound']) { //deleted post
$description .= 'Quoted post deleted.';
} elseif (isset($replyQuotedRecord['detached']) && $replyQuotedRecord['detached']) { //detached quote
$uri_explode = explode('/', $replyQuotedRecord['uri']);
$uri_reconstructed = self::URI . '/profile/' . $uri_explode[2] . '/post/' . $uri_explode[4];
$description .= '<a href="' . $uri_reconstructed . '">Quoted post detached.</a>';
} elseif (isset($replyQuotedRecord['blocked']) && $replyQuotedRecord['blocked']) { //blocked by quote author
$description .= 'Author of quoted post has blocked OP.';
} elseif (
($replyQuotedRecord['$type'] ?? '') === 'app.bsky.feed.defs#generatorView' ||
($replyQuotedRecord['$type'] ?? '') === 'app.bsky.graph.defs#listView'
) {
$description .= $this->getListFeedDescription($replyQuotedRecord);
} elseif (
($replyQuotedRecord['$type'] ?? '') === 'app.bsky.graph.starterpack' ||
($replyQuotedRecord['$type'] ?? '') === 'app.bsky.graph.defs#starterPackViewBasic'
) {
$description .= $this->getStarterPackDescription($replyPost['embed']['record']);
} else {
$quotedAuthorDid = $replyQuotedRecord['author']['did'];
$quotedDisplayName = $replyQuotedRecord['author']['displayName'] ?? '';
$quotedDisplayName = e($quotedDisplayName);
$quotedAuthorHandle = $replyQuotedRecord['author']['handle'] !== 'handle.invalid' ? '<i>@' . $replyQuotedRecord['author']['handle'] . '</i>' : '';
$parts = explode('/', $replyQuotedRecord['uri']);
$quotedPostId = end($parts);
$quotedPostUri = self::URI . '/profile/' . $this->fallbackAuthor($replyQuotedRecord['author'], 'url') . '/post/' . $quotedPostId;
$parts = explode('/', $replyQuotedRecord['uri']);
$quotedPostId = end($parts);
$quotedPostUri = self::URI . '/profile/' . $this->fallbackAuthor($replyQuotedRecord['author'], 'url') . '/post/' . $quotedPostId;
//quoted post - post
$description .= $this->getPostDescription(
$quotedDisplayName,
$quotedAuthorHandle,
$quotedPostUri,
$replyQuotedRecord,
'quote'
);
//quoted post - post
$description .= $this->getPostDescription(
$quotedDisplayName,
$quotedAuthorHandle,
$quotedPostUri,
$replyQuotedRecord,
'quote'
);
if (isset($replyQuotedRecord['value']['embed']['$type'])) {
//quoted post - post link embed
if ($replyQuotedRecord['value']['embed']['$type'] === 'app.bsky.embed.external') {
$description .= $this->parseExternal($replyQuotedRecord['value']['embed']['external'], $quotedAuthorDid);
}
if (isset($replyQuotedRecord['value']['embed']['$type'])) {
//quoted post - post link embed
if ($replyQuotedRecord['value']['embed']['$type'] === 'app.bsky.embed.external') {
$description .= $this->parseExternal($replyQuotedRecord['value']['embed']['external'], $quotedAuthorDid);
}
//quoted post - post video
if (
$replyQuotedRecord['value']['embed']['$type'] === 'app.bsky.embed.video' ||
(
$replyQuotedRecord['value']['embed']['$type'] === 'app.bsky.embed.recordWithMedia' &&
$replyQuotedRecord['value']['embed']['media']['$type'] === 'app.bsky.embed.video'
)
) {
$description .= $this->getPostVideoDescription(
$replyQuotedRecord['value']['embed']['video'] ?? $replyQuotedRecord['value']['embed']['media']['video'],
$quotedAuthorDid
);
}
//quoted post - post video
if (
$replyQuotedRecord['value']['embed']['$type'] === 'app.bsky.embed.video' ||
(
$replyQuotedRecord['value']['embed']['$type'] === 'app.bsky.embed.recordWithMedia' &&
$replyQuotedRecord['value']['embed']['media']['$type'] === 'app.bsky.embed.video'
)
) {
$description .= $this->getPostVideoDescription(
$replyQuotedRecord['value']['embed']['video'] ?? $replyQuotedRecord['value']['embed']['media']['video'],
$quotedAuthorDid
);
}
//quoted post - post images
if (
$replyQuotedRecord['value']['embed']['$type'] === 'app.bsky.embed.images' ||
(
$replyQuotedRecord['value']['embed']['$type'] === 'app.bsky.embed.recordWithMedia' &&
$replyQuotedRecord['value']['embed']['media']['$type'] === 'app.bsky.embed.images'
)
) {
foreach ($replyQuotedRecord['embeds'] as $embed) {
if (
$embed['$type'] === 'app.bsky.embed.images#view' ||
($embed['$type'] === 'app.bsky.embed.recordWithMedia#view' && $embed['media']['$type'] === 'app.bsky.embed.images#view')
) {
$images = $embed['images'] ?? $embed['media']['images'];
foreach ($images as $image) {
$description .= $this->getPostImageDescription($image);
//quoted post - post images
if (
$replyQuotedRecord['value']['embed']['$type'] === 'app.bsky.embed.images' ||
(
$replyQuotedRecord['value']['embed']['$type'] === 'app.bsky.embed.recordWithMedia' &&
$replyQuotedRecord['value']['embed']['media']['$type'] === 'app.bsky.embed.images'
)
) {
foreach ($replyQuotedRecord['embeds'] as $embed) {
if (
$embed['$type'] === 'app.bsky.embed.images#view' ||
($embed['$type'] === 'app.bsky.embed.recordWithMedia#view' && $embed['media']['$type'] === 'app.bsky.embed.images#view')
) {
$images = $embed['images'] ?? $embed['media']['images'];
foreach ($images as $image) {
$description .= $this->getPostImageDescription($image);
}
}
}
}
}
}
$description .= '</p>';
}
$description .= '</p>';
}
}
@@ -496,7 +500,7 @@ class BlueskyBridge extends BridgeAbstract
$videoMime = $video['mimeType'];
$thumbnail = "poster=\"https://video.bsky.app/watch/$authorDID/$videoCID/thumbnail.jpg\"" ?? '';
$videoURL = "https://bsky.social/xrpc/com.atproto.sync.getBlob?did=$authorDID&cid=$videoCID";
return "<figure><video loop $thumbnail controls src=\"$videoURL\" type=\"$videoMime\"/></figure>";
return "<figure><video loop $thumbnail preload=\"none\" controls src=\"$videoURL\" type=\"$videoMime\"/></figure>";
}
private function getPostImageDescription(array $image)
@@ -606,9 +610,9 @@ class BlueskyBridge extends BridgeAbstract
private function getAuthorFeed($did, $filter)
{
$uri = 'https://public.api.bsky.app/xrpc/app.bsky.feed.getAuthorFeed?actor=' . urlencode($did) . '&filter=' . urlencode($filter) . '&limit=30';
if (Debug::isEnabled()) {
$this->logger->debug($uri);
}
$this->logger->debug($uri);
$response = json_decode(getContents($uri), true);
return $response;
}

View File

@@ -98,7 +98,7 @@ class BugzillaBridge extends BridgeAbstract
// Array of comments is here
if (!isset($json['bugs'][$this->bugid]['comments'])) {
returnClientError('Cannot find REST endpoint');
throwClientException('Cannot find REST endpoint');
}
foreach ($json['bugs'][$this->bugid]['comments'] as $comment) {
@@ -131,7 +131,7 @@ class BugzillaBridge extends BridgeAbstract
// Array of changesets which contain an array of changes
if (!isset($json['bugs']['0']['history'])) {
returnClientError('Cannot find REST endpoint');
throwClientException('Cannot find REST endpoint');
}
foreach ($json['bugs']['0']['history'] as $changeset) {

View File

@@ -30,7 +30,7 @@ URI;
// Build the URL from the first anchor element. The list is sorted by year, descending, so the first element is the current year.
$firstAnchor = $html->find('a', 0)
or returnServerError('Could not find the proper HTML element.');
or throwServerException('Could not find the proper HTML element.');
$url = $firstAnchor->href;
@@ -38,7 +38,7 @@ URI;
$html = getSimpleHTMLDOMCached($url, self::CACHE_TIMEOUT);
$rows = $html->find('table.table > tbody > tr')
or returnServerError('Could not find the proper HTML elements.');
or throwServerException('Could not find the proper HTML elements.');
foreach ($rows as $row) {
$item = $this->generateItemFromRow($row);

View File

@@ -50,7 +50,7 @@ class CNETBridge extends SitemapBridge
}
if (empty($links)) {
returnClientError('Failed to retrieve article list');
throwClientException('Failed to retrieve article list');
}
foreach ($links as $article_uri) {

View File

@@ -87,7 +87,7 @@ class CVEDetailsBridge extends BridgeAbstract
$vendor = $html->find('#contentdiv h1 > a', 0);
if ($vendor == null) {
returnServerError('Invalid Vendor ID ' . $this->getInput('vendor_id') . ' or Product ID ' . $this->getInput('product_id'));
throwServerException('Invalid Vendor ID ' . $this->getInput('vendor_id') . ' or Product ID ' . $this->getInput('product_id'));
}
$this->vendor = $vendor->innertext;

View File

@@ -72,14 +72,14 @@ class CachetBridge extends BridgeAbstract
{
$ping = getContents(urljoin($this->getURI(), '/api/v1/ping'));
if (!$this->validatePing($ping)) {
returnClientError('Provided URI is invalid!');
throwClientException('Provided URI is invalid!');
}
$url = urljoin($this->getURI(), '/api/v1/incidents?sort=id&order=desc');
$incidents = getContents($url);
$incidents = json_decode($incidents);
if ($incidents === null) {
returnClientError('/api/v1/incidents returned no valid json');
throwClientException('/api/v1/incidents returned no valid json');
}
usort($incidents->data, function ($a, $b) {

View File

@@ -36,7 +36,7 @@ class CastorusBridge extends BridgeAbstract
$title = $activity->find('a', 0);
if (!$title) {
returnServerError('Cannot find title!');
throwServerException('Cannot find title!');
}
return trim($title->plaintext);
@@ -48,7 +48,7 @@ class CastorusBridge extends BridgeAbstract
$url = $activity->find('a', 0);
if (!$url) {
returnServerError('Cannot find url!');
throwServerException('Cannot find url!');
}
return self::URI . $url->href;
@@ -62,7 +62,7 @@ class CastorusBridge extends BridgeAbstract
$nodes = $activity->find('*');
if (!$nodes) {
returnServerError('Cannot find nodes!');
throwServerException('Cannot find nodes!');
}
foreach ($nodes as $node) {
@@ -78,7 +78,7 @@ class CastorusBridge extends BridgeAbstract
$price = $activity->find('span', 1);
if (!$price) {
returnServerError('Cannot find price!');
throwServerException('Cannot find price!');
}
return $price->innertext;
@@ -92,13 +92,13 @@ class CastorusBridge extends BridgeAbstract
$html = getSimpleHTMLDOM(self::URI);
if (!$html) {
returnServerError('Could not load data from ' . self::URI . '!');
throwServerException('Could not load data from ' . self::URI . '!');
}
$activities = $html->find('div#activite > li');
if (!$activities) {
returnServerError('Failed to find activities!');
throwServerException('Failed to find activities!');
}
foreach ($activities as $activity) {

View File

@@ -72,15 +72,9 @@ class CentreFranceBridge extends BridgeAbstract
$newspaperUrl = 'https://www.' . $this->getInput('newspaper') . '/' . $localitySlug . '/';
$html = getSimpleHTMLDOM($newspaperUrl);
// Articles are detected through their titles
foreach ($html->find('.c-titre') as $articleTitleDOMElement) {
$articleLinkDOMElement = $articleTitleDOMElement->find('a', 0);
// Ignore articles in the « Les + partagés » block
if (strpos($articleLinkDOMElement->id, 'les_plus_partages') !== false) {
continue;
}
// Articles are detected through a standard tag
foreach ($html->find('article') as $articleDOMElement) {
$articleLinkDOMElement = $articleDOMElement->find('a', 0);
$articleURI = $articleLinkDOMElement->href;
// If the URI has already been processed, ignore it
@@ -96,7 +90,7 @@ class CentreFranceBridge extends BridgeAbstract
$articleTitle = '';
// If article is reserved for subscribers
if ($articleLinkDOMElement->find('span.premium-picto', 0)) {
if ($articleLinkDOMElement->find('span.premium-icon', 0)) {
if ($this->getInput('remove-reserved-for-subscribers-articles') === true) {
continue;
}
@@ -104,18 +98,22 @@ class CentreFranceBridge extends BridgeAbstract
$articleTitle .= '🔒 ';
}
$articleTitleDOMElement = $articleLinkDOMElement->find('span[data-tb-title]', 0);
if ($articleTitleDOMElement === null) {
continue;
}
if ($limit > 0 && count($this->items) === $limit) {
break;
}
$articleTitle .= $articleLinkDOMElement->find('span[data-tb-title]', 0)->innertext;
$articleFullURI = urljoin('https://www.' . $this->getInput('newspaper') . '/', $articleURI);
// Loop through each possible title class name
for ($i = 1; $i <= 3; $i++) {
$articleTitleDOMElement = $articleLinkDOMElement->find('.typo-card-title-' . $i, 0);
if (!$articleTitleDOMElement instanceof \simple_html_dom_node) {
continue;
}
$articleTitle .= $articleTitleDOMElement->text();
break;
}
$articleFullURI = urljoin('https://www.' . $this->getInput('newspaper') . '/', $articleURI);
$item = [
'title' => $articleTitle,
'uri' => $articleFullURI,
@@ -184,7 +182,7 @@ class CentreFranceBridge extends BridgeAbstract
$articleTags = $html->find('#content>div.flex+div.grid section>.bg-gray-light>a.border-gray-dark');
if (is_array($articleTags)) {
$item['categories'] = array_map(static fn ($articleTag) => $articleTag->innertext, $articleTags);
$item['categories'] = array_map(static fn ($articleTag) => html_entity_decode($articleTag->innertext), $articleTags);
}
$explode = explode('_', $uri);
@@ -195,6 +193,10 @@ class CentreFranceBridge extends BridgeAbstract
$item['uid'] = $uid;
}
if (!isset($item['content'])) {
$item['content'] = '';
}
// If the article is a "grand format", we use another parsing strategy
if ($item['content'] === '' && $html->find('article') !== []) {
$articleContent = $html->find('article > section');

View File

@@ -24,7 +24,7 @@ class CeskaTelevizeBridge extends BridgeAbstract
$validUrl = '/^(https:\/\/www\.ceskatelevize\.cz\/porady\/\d+-[a-z0-9-]+\/)(bonus\/)?$/';
if (!preg_match($validUrl, $url, $match)) {
returnServerError('Invalid url');
throwServerException('Invalid url');
}
$category = $match[4] ?? 'nove';
@@ -63,7 +63,7 @@ class CeskaTelevizeBridge extends BridgeAbstract
} elseif (strpos($string, 'včera') !== false) {
return strtotime('yesterday');
} elseif (!preg_match('/(\d+).(\d+).((\d+))?/', $string, $match)) {
returnServerError('Could not get date from Česká televize string');
throwServerException('Could not get date from Česká televize string');
}
$date = sprintf('%04d-%02d-%02d', $match[3] ?? date('Y'), $match[2], $match[1]);

186
bridges/ComickBridge.php Normal file
View File

@@ -0,0 +1,186 @@
<?php
class ComickBridge extends BridgeAbstract
{
const MAINTAINER = 'phantop';
const NAME = 'Comick';
const URI = 'https://comick.io/';
const DESCRIPTION = 'Returns the latest chapters for a manga on comick.io.';
const PARAMETERS = [[
'slug' => [
'name' => 'Manga Slug',
'type' => 'text',
'required' => true,
'title' => 'The part of the URL after /comic/',
'exampleValue' => '00-kusuriya-no-hitorigoto-maomao-no-koukyuu-nazotoki-techou'
],
'lang' => [
'name' => 'Language',
'type' => 'list',
'title' => 'Language for comic (list is # of comics, descending)',
'values' => [
'English' => 'en',
'Brazilian Portuguese' => 'pt-br',
'Spanish Latin American' => 'es-la',
'Russian' => 'ru',
'Vietnamese' => 'vi',
'French' => 'fr',
'Polish' => 'pl',
'Indonesian' => 'id',
'Turkish' => 'tr',
'Italian' => 'it',
'Spanish; Castilian' => 'es',
'Ukrainian' => 'uk',
'Arabic' => 'ar',
'Hong Kong (Traditional Chinese)' => 'zh-hk',
'Hungarian' => 'hu',
'Chinese' => 'zh',
'German' => 'de',
'Korean' => 'ko',
'Thai' => 'th',
'Catalan; Valencian' => 'ca',
'Bulgarian' => 'bg',
'Persian' => 'fa',
'Romanian, Moldavian, Moldovan' => 'ro',
'Czech' => 'cs',
'Mongolian' => 'mn',
'Portuguese' => 'pt',
'Hebrew (modern)' => 'he',
'Hindi' => 'hi',
'Filipino/Tagalog' => 'tl',
'Finnish' => 'fi',
'Malay' => 'ms',
'Basque' => 'eu',
'Kazakh' => 'kk',
'Serbian' => 'sr',
'Burmese' => 'my',
'Japanese' => 'ja',
'Greek, Modern' => 'el',
'Dutch' => 'nl',
'Bengali' => 'bn',
'Uzbek' => 'uz',
'Esperanto' => 'eo',
'Lithuanian' => 'lt',
'Georgian' => 'ka',
'Danish' => 'da',
'Tamil' => 'ta',
'Swedish' => 'sv',
'Belarusian' => 'be',
'Chuvash' => 'cv',
'Croatian' => 'hr',
'Latin' => 'la',
'Nepali' => 'ne',
'Urdu' => 'ur',
'Galician' => 'gl',
'Norwegian' => 'no',
'Albanian' => 'sq',
'Irish' => 'ga',
'Javanese' => 'jv',
'Telugu' => 'te',
'Slovene' => 'sl',
'Estonian' => 'et',
'Azerbaijani' => 'az',
'Slovak' => 'sk',
'Afrikaans' => 'af',
'Latvian' => 'lv',
],
'defaultValue' => 'en'
],
'fetch' => [
'name' => 'Fetch chapter page images',
'type' => 'list',
'title' => 'Places chapter images in feed contents. Entries will consume more bandwidth.',
'defaultValue' => 'c',
'values' => [
'None' => 'n',
'Content' => 'c',
'Enclosure' => 'e'
]
],
'limit' => [
'name' => 'Limit',
'type' => 'number',
'title' => 'Maximum number of chapters to return',
'defaultValue' => 10
]
]];
private $title;
private function getComick($url)
{
$API = 'https://api.comick.fun';
// Need a non-cURL UA, otherwise we get Cloudflare 403'd
$opts = [
CURLOPT_USERAGENT => 'rss-bridge (https://github.com/RSS-Bridge/rss-bridge)'
];
$content = getContents("$API/$url", [], $opts);
return json_decode($content, true);
}
public function collectData()
{
$slug = $this->getInput('slug');
$lang = $this->getInput('lang');
$limit = $this->getInput('limit');
$manga = $this->getComick("comic/$slug");
$hid = $manga['comic']['hid'];
$this->title = $manga['comic']['title'];
$manga = $this->getComick("comic/$hid/chapters?lang=$lang&limit=$limit");
foreach ($manga['chapters'] as $chapter) {
$hid = $chapter['hid'];
$item['author'] = implode(', ', $chapter['group_name']);
$item['timestamp'] = strtotime($chapter['created_at']);
$item['uri'] = $this->getURI() . '/' . $hid;
$item['title'] = '';
if ($chapter['vol']) {
$item['title'] .= ' Vol. ' . $chapter['vol'];
}
if ($chapter['chap']) {
$item['title'] .= ' Ch. ' . $chapter['chap'];
}
if ($chapter['title']) {
$item['title'] .= ' - ' . $chapter['title'];
}
if ($this->getInput('fetch') != 'n') {
$chapter = $this->getComick("chapter/$hid");
if (isset($chapter['chapter']['md_images'])) {
$item['content'] = '';
foreach ($chapter['chapter']['md_images'] as $image) {
$img = 'https://meo.comick.pictures/' . $image['b2key'];
if ($this->getInput('fetch') == 'c') {
$item['content'] .= '<img src="' . $img . '" />';
}
if ($this->getInput('fetch') == 'e') {
$item['enclosures'][] = $img;
}
}
}
}
$this->items[] = $item;
}
}
public function getName()
{
if ($this->title) {
return parent::getName() . ' - ' . $this->title;
}
return parent::getName();
}
public function getURI()
{
if ($this->getInput('slug')) {
return self::URI . 'comic/' . $this->getInput('slug');
}
return parent::getURI();
}
}

View File

@@ -217,7 +217,7 @@ class CssSelectorBridge extends BridgeAbstract
$links = $page->find($url_selector);
if (empty($links)) {
returnClientError('No results for URL selector');
throwClientException('No results for URL selector');
}
$link_to_item = [];
@@ -245,13 +245,13 @@ class CssSelectorBridge extends BridgeAbstract
}
if (empty($link_to_item)) {
returnClientError('The provided URL selector matches some elements, but they do not contain links.');
throwClientException('The provided URL selector matches some elements, but they do not contain links.');
}
$links = $this->filterUrlList(array_keys($link_to_item), $url_pattern, $limit);
if (empty($links)) {
returnClientError('No results for URL pattern');
throwClientException('No results for URL pattern');
}
$items = [];
@@ -274,7 +274,7 @@ class CssSelectorBridge extends BridgeAbstract
protected function expandEntryWithSelector($entry_url, $content_selector, $content_cleanup = null, $title_cleanup = null, $title_default = null)
{
if (empty($content_selector)) {
returnClientError('Please specify a content selector');
throwClientException('Please specify a content selector');
}
$entry_html = getSimpleHTMLDOMCached($entry_url);

View File

@@ -187,7 +187,7 @@ class CssSelectorComplexBridge extends BridgeAbstract
// Fetch the elements from the article pages.
if ($use_article_pages) {
if (empty($article_page_content_selector)) {
returnClientError('`Article selector` is required when `Load article page` is enabled');
throwClientException('`Article selector` is required when `Load article page` is enabled');
}
foreach (array_keys($entry_elements) as $uri) {
@@ -307,7 +307,7 @@ class CssSelectorComplexBridge extends BridgeAbstract
$entryElements = $page->find($entry_selector);
if (empty($entryElements)) {
returnClientError('No entry elements for entry selector');
throwClientException('No entry elements for entry selector');
}
// Extract URIs with the associated entry element
@@ -327,7 +327,7 @@ class CssSelectorComplexBridge extends BridgeAbstract
}
if (empty($links_with_elements)) {
returnClientError('The provided URL selector matches some elements, but they do not
throwClientException('The provided URL selector matches some elements, but they do not
contain links.');
}
@@ -335,7 +335,7 @@ class CssSelectorComplexBridge extends BridgeAbstract
$filtered_urls = $this->filterUrlList(array_keys($links_with_elements), $url_pattern, $limit);
if (empty($filtered_urls)) {
returnClientError('No results for URL pattern');
throwClientException('No results for URL pattern');
}
$items = [];
@@ -359,7 +359,7 @@ class CssSelectorComplexBridge extends BridgeAbstract
$article_content = $entry_html->find($content_selector, 0);
if (is_null($article_content)) {
returnClientError('Could not get article content at URL: ' . $entry_url);
throwClientException('Could not get article content at URL: ' . $entry_url);
}
$article_content = defaultLinkTo($article_content, $entry_url);
@@ -370,7 +370,7 @@ class CssSelectorComplexBridge extends BridgeAbstract
{
$date = date_parse_from_format($format, $timeStr);
if ($date['error_count'] != 0) {
returnClientError('Error while parsing time string');
throwClientException('Error while parsing time string');
}
$timestamp = mktime(
@@ -383,7 +383,7 @@ class CssSelectorComplexBridge extends BridgeAbstract
);
if ($timestamp == false) {
returnClientError('Error while creating timestamp');
throwClientException('Error while creating timestamp');
}
return $timestamp;

View File

@@ -15,7 +15,7 @@ class CubariProxyBridge extends BridgeAbstract
'MangAventure' => 'mangadventure',
'MangaDex' => 'mangadex',
'MangaKatana' => 'mangakatana',
'MangaSee' => 'mangasee',
'WeebCentral' => 'weebcentral',
]
],
'series' => [

114
bridges/CybernewsBridge.php Normal file
View File

@@ -0,0 +1,114 @@
<?php
declare(strict_types=1);
class CybernewsBridge extends BridgeAbstract
{
const NAME = 'Cybernews';
const URI = 'https://cybernews.com';
const DESCRIPTION = 'Fetches the latest news from Cybernews';
const MAINTAINER = 'tillcash';
const CACHE_TIMEOUT = 3600; // 1 hour
const MAX_ARTICLES = 5;
public function collectData()
{
$sitemapXml = getContents(self::URI . '/news-sitemap.xml');
if (!$sitemapXml) {
throwServerException('Unable to retrieve Cybernews sitemap');
}
$sitemap = simplexml_load_string($sitemapXml, null, LIBXML_NOCDATA);
if (!$sitemap) {
throwServerException('Unable to parse Cybernews sitemap');
}
foreach ($sitemap->url as $entry) {
$url = trim((string) $entry->loc);
$lastmod = trim((string) $entry->lastmod);
if (!$url) {
continue;
}
$pathParts = explode('/', trim(parse_url($url, PHP_URL_PATH), '/'));
$category = isset($pathParts[0]) && $pathParts[0] !== '' ? $pathParts[0] : '';
// Skip non-English versions
if (in_array($category, ['nl', 'de'], true)) {
continue;
}
$namespaces = $entry->getNamespaces(true);
$title = '';
if (isset($namespaces['news'])) {
$news = $entry->children($namespaces['news'])->news;
if ($news) {
$title = trim((string) $news->title);
}
}
if (!$title) {
continue;
}
$this->items[] = [
'title' => $title,
'uri' => $url,
'uid' => $url,
'timestamp' => strtotime($lastmod),
'categories' => $category ? [$category] : [],
'content' => $this->fetchFullArticle($url),
];
if (count($this->items) >= self::MAX_ARTICLES) {
break;
}
}
}
private function fetchFullArticle(string $url): string
{
$html = getSimpleHTMLDOMCached($url);
if (!$html) {
return 'Unable to fetch article content';
}
$article = $html->find('article', 0);
if (!$article) {
return 'Unable to parse article content';
}
// Remove unnecessary elements
$removeSelectors = [
'script',
'style',
'div.links-bar',
'div.google-news-cta',
'div.a-wrapper',
'div.embed_youtube',
];
foreach ($removeSelectors as $selector) {
foreach ($article->find($selector) as $element) {
$element->outertext = '';
}
}
// Handle lazy-loaded images
foreach ($article->find('img') as $img) {
if (!empty($img->{'data-src'})) {
$img->src = $img->{'data-src'};
unset($img->{'data-src'});
}
}
return $article->innertext;
}
}

View File

@@ -37,6 +37,26 @@ class DRKBlutspendeBridge extends FeedExpander
]
];
const OFFER_LOW_PRIORITIES = [
'Imbiss nach der Blutspende',
'Registrierung als Stammzellspender',
'Typisierung möglich!',
'Allgemeine Informationen',
'Krankenkassen belohnen Blutspender',
'Wer benötigt eigentlich eine Blutspende?',
'Win-Win-Situation für die Gesundheit!',
'Terminreservierung',
'Du möchtest das erste Mal Blut spenden?',
'Spende-Check',
'Sie haben Fragen vor Ihrer Blutspende?'
];
const IMAGE_PRIORITIES = [
'DRK',
'Imbiss',
'Obst',
];
public function collectData()
{
$limitItems = intval($this->getInput('limit_items'));
@@ -45,37 +65,116 @@ class DRKBlutspendeBridge extends FeedExpander
protected function parseItem(array $item)
{
$html = getSimpleHTMLDOM($item['uri']);
$html = getSimpleHTMLDOMCached($item['uri']);
$detailsElement = $html->find('.details', 0);
$dateElement = $detailsElement->find('.datum', 0);
$dateLines = self::explodeLines($dateElement->plaintext);
$addressElement = $detailsElement->find('.adresse', 0);
$addressLines = self::explodeLines($addressElement->plaintext);
$dateLines = self::explodeLines($detailsElement->find('.datum', 0)->plaintext);
$addressLines = self::explodeLines($detailsElement->find('.adresse', 0)->plaintext);
$infoElement = $detailsElement->find('.angebote > h4 + p', 0);
$info = $infoElement ? $infoElement->innertext : '';
$info = $infoElement ? trim($infoElement->plaintext) : '';
$imageElements = $detailsElement->find('.fotos img');
$offers = self::parseOffers($detailsElement->find('.angebote .item'));
$item['title'] = $dateLines[0] . ' ' . $dateLines[1] . ' ' . $addressLines[0] . ' - ' . $addressLines[1];
$images = self::parseImages($detailsElement->find('.fotos', 0));
usort($images, function ($imageA, $imageB): int {
list($titleA) = $imageA;
list($titleB) = $imageB;
$prioA = 0;
$prioB = 0;
foreach (self::IMAGE_PRIORITIES as $prioIndex => $prioTitleNeedle) {
if (stripos($titleA, $prioTitleNeedle) !== false) {
$prioA = $prioIndex + 1;
}
if (stripos($titleB, $prioTitleNeedle) !== false) {
$prioB = $prioIndex + 1;
}
}
return $prioA - $prioB;
});
$item['content'] = <<<HTML
<p><b>{$dateLines[0]} {$dateLines[1]}</b></p>
<p>{$addressElement->innertext}</p>
<p>{$info}</p>
$itemContent = <<<HTML
<div>
<p>
<b>{$dateLines[0]} {$dateLines[1]}</b><br>
{$addressLines[3]}
</p>
<p>
<b>{$addressLines[0]}</b><br>
{$addressLines[1]}<br>
{$addressLines[2]}
</p>
</div>
HTML;
foreach ($imageElements as $imageElement) {
$src = $imageElement->getAttribute('src');
$item['content'] .= <<<HTML
<p><img src="{$src}"></p>
if ($info) {
$itemContent .= <<<HTML
<div>
<h3>Infos</h3>
<p>{$info}</p>
</div>
HTML;
}
$majorOffers = array_filter($offers, fn($title): bool => !in_array($title, self::OFFER_LOW_PRIORITIES), ARRAY_FILTER_USE_KEY);
foreach ($majorOffers as $offerTitle => list($offerText, $offerImages)) {
$itemContent .= <<<HTML
<div>
<h3>{$offerTitle}</h3>
<p>{$offerText}</p>
HTML;
foreach ($offerImages as list($imageTitle, $imageUrl)) {
$itemContent .= <<<HTML
<figure>
<img src="{$imageUrl}">
<figcaption>{$imageTitle}</figcaption>
</figure>
HTML;
}
$itemContent .= <<<HTML
</div>
HTML;
}
if (count($images) > 0) {
$itemContent .= <<<HTML
<div>
<h3>Fotos</h3>
HTML;
foreach ($images as list($imageTitle, $imageUrl)) {
$itemContent .= <<<HTML
<figure>
<img src="{$imageUrl}">
<figcaption>{$imageTitle}</figcaption>
</figure>
HTML;
}
$itemContent .= <<<HTML
</div>
HTML;
}
$minorOffers = array_filter($offers, fn($title): bool => in_array($title, self::OFFER_LOW_PRIORITIES), ARRAY_FILTER_USE_KEY);
foreach ($minorOffers as $offerTitle => list($offerText)) {
$itemContent .= <<<HTML
<div>
<h3>{$offerTitle}</h3>
<p>{$offerText}</p>
</div>
HTML;
}
$item['title'] = $dateLines[0] . ' ' . $dateLines[1] . ' ' . $addressLines[0] . ' - ' . $addressLines[1];
$item['content'] = $itemContent;
$item['description'] = null;
$item['enclosures'] = array_map(
function ($image): string {
list($title, $url) = $image;
return $url . '#' . urlencode(str_replace(' ', '_', $title));
},
$images
);
return $item;
}
@@ -97,6 +196,67 @@ class DRKBlutspendeBridge extends FeedExpander
return self::BASE_URI . '/blutspendetermine/termine.rss?date_to=' . $dateTo . '&radius=' . $radius . '&term=' . $term;
}
private function parseImages($parentElement): array
{
$images = [];
if ($parentElement) {
$elements = $parentElement->find('a[data-lightbox]');
foreach ($elements as $i => $element) {
$url = trim($element->getAttribute('href'));
if (!$url) {
continue;
}
$title = trim($element->getAttribute('title'));
if (!$title) {
$number = $i + 1;
$title = "Foto {$number}";
}
$images[] = [$title, $url];
}
}
return $images;
}
private function parseOffers($offerElements): array
{
$offers = [];
foreach ($offerElements as $element) {
$title = self::getCleanPlainText($element->find(':is(h1,h2,h3,h4,h5,h6)', 0));
$text = trim(substr(self::getCleanPlainText($element), strlen($title)));
if (!$title || !$text) {
continue;
}
$linkElements = $element->find('a');
foreach ($linkElements as $linkElement) {
$linkText = trim($linkElement->plaintext);
$linkUrl = trim($linkElement->getAttribute('href'));
if (!$linkText || !$linkUrl) {
continue;
}
$linkHtml = <<<HTML
<a href="{$linkUrl}" target="_blank">{$linkText}</a>
HTML;
$text = str_replace($linkText, $linkHtml, $text);
}
$offers[$title] = [$text, self::parseImages($element)];
}
return $offers;
}
private function getCleanPlainText($htmlElement): string
{
return $htmlElement ? trim(preg_replace('/\s+/', ' ', html_entity_decode($htmlElement->plaintext))) : '';
}
/**
* Returns an array of strings, each of which is a substring of string formed by splitting it on boundaries formed by line breaks.
*/

View File

@@ -44,68 +44,32 @@ class DailymotionBridge extends BridgeAbstract
public function getIcon()
{
return 'https://static1-ssl.dmcdn.net/images/neon/favicons/android-icon-36x36.png.vf806ca4ed0deed812';
return 'https://static1.dmcdn.net/neon-user-ssr/prod/favicons/apple-icon-60x60.831b96ed0a8eca7f6539.png';
}
public function collectData()
{
if ($this->queriedContext === 'By username' || $this->queriedContext === 'By playlist id') {
$apiJson = getContents($this->getApiUrl());
$apiData = json_decode($apiJson, true);
$apiJson = getContents($this->getApiUrl());
$apiData = json_decode($apiJson, true);
if ($this->queriedContext === 'By playlist id') {
$this->feedName = $this->getPlaylistTitle($this->getInput('p'));
foreach ($apiData['list'] as $apiItem) {
$item = [];
$item['uri'] = $apiItem['url'];
$item['uid'] = $apiItem['id'];
$item['title'] = $apiItem['title'];
$item['timestamp'] = $apiItem['created_time'];
$item['author'] = $apiItem['owner.screenname'];
$item['content'] = '<p><a href="' . $apiItem['url'] . '">
<img src="' . $apiItem['thumbnail_url'] . '"></a></p><p>' . $apiItem['description'] . '</p>';
$item['categories'] = $apiItem['tags'];
$item['enclosures'][] = $apiItem['thumbnail_url'];
$this->items[] = $item;
}
}
if ($this->queriedContext === 'From search results') {
$html = getSimpleHTMLDOM($this->getURI());
foreach ($apiData['list'] as $apiItem) {
$item = [];
foreach ($html->find('div.media a.preview_link') as $element) {
$item = [];
$item['uri'] = $apiItem['url'];
$item['uid'] = $apiItem['id'];
$item['title'] = $apiItem['title'];
$item['timestamp'] = $apiItem['created_time'];
$item['author'] = $apiItem['owner.screenname'];
$item['content'] = '<p><a href="' . $apiItem['url'] . '">
<img src="' . $apiItem['thumbnail_url'] . '"></a></p><p>' . $apiItem['description'] . '</p>';
$item['categories'] = $apiItem['tags'];
$item['enclosures'][] = $apiItem['thumbnail_url'];
$item['id'] = str_replace('/video/', '', strtok($element->href, '_'));
$metadata = $this->getMetadata($item['id']);
if (empty($metadata)) {
continue;
}
$item['uri'] = $metadata['uri'];
$item['title'] = $metadata['title'];
$item['timestamp'] = $metadata['timestamp'];
$item['content'] = '<a href="'
. $item['uri']
. '"><img src="'
. $metadata['thumbnailUri']
. '" /></a><br><a href="'
. $item['uri']
. '">'
. $item['title']
. '</a>';
$this->items[] = $item;
if (count($this->items) >= 5) {
break;
}
}
$this->items[] = $item;
}
}
@@ -136,6 +100,7 @@ class DailymotionBridge extends BridgeAbstract
public function getURI()
{
$uri = self::URI;
switch ($this->queriedContext) {
case 'By username':
$uri .= 'user/' . urlencode($this->getInput('u'));
@@ -162,35 +127,11 @@ class DailymotionBridge extends BridgeAbstract
return $uri;
}
private function getMetadata($id)
{
$metadata = [];
$html = getSimpleHTMLDOM(self::URI . 'video/' . $id);
if (!$html) {
return $metadata;
}
$metadata['title'] = $html->find('meta[property=og:title]', 0)->getAttribute('content');
$metadata['timestamp'] = strtotime(
$html->find('meta[property=video:release_date]', 0)->getAttribute('content')
);
$metadata['thumbnailUri'] = $html->find('meta[property=og:image]', 0)->getAttribute('content');
$metadata['uri'] = $html->find('meta[property=og:url]', 0)->getAttribute('content');
return $metadata;
}
private function getPlaylistTitle($id)
{
$title = '';
$url = self::URI . 'playlist/' . $id;
$html = getSimpleHTMLDOM($url);
$title = $html->find('meta[property=og:title]', 0)->getAttribute('content');
return $title;
$apiJson = getContents($this->apiUrl . '/playlist/' . $this->getInput('p'));
$apiData = json_decode($apiJson, true);
return $apiData['name'];
}
private function getApiUrl()
@@ -204,6 +145,9 @@ class DailymotionBridge extends BridgeAbstract
return $this->apiUrl . '/playlist/' . $this->getInput('p')
. '/videos?fields=' . urlencode($this->apiFields) . '&limit=5';
break;
case 'From search results':
return $this->apiUrl . '/videos?search=' . $this->getInput('s') . '&fields=' . urlencode($this->apiFields) . '&limit=5';
break;
}
}
}

View File

@@ -1,30 +0,0 @@
<?php
class DansTonChatBridge extends BridgeAbstract
{
const MAINTAINER = 'Astalaseven';
const NAME = 'DansTonChat Bridge';
const URI = 'https://danstonchat.com/';
const CACHE_TIMEOUT = 21600; //6h
const DESCRIPTION = 'Returns latest quotes from DansTonChat.';
public function collectData()
{
$url = self::URI . 'latest.html';
$dom = getSimpleHTMLDOM($url);
$items = $dom->find('div.item');
foreach ($items as $element) {
$item = [];
$item['uri'] = $element->find('a', 0)->href;
$titleContent = $element->find('h3 a', 0);
if ($titleContent) {
$item['title'] = 'DansTonChat ' . html_entity_decode($titleContent->plaintext, ENT_QUOTES);
} else {
$item['title'] = 'DansTonChat';
}
$item['content'] = $element->find('div.item-content a', 0)->innertext;
$this->items[] = $item;
}
}
}

View File

@@ -75,7 +75,7 @@ apple-icon-5c6fa9f2bce280428589c6195b7f1924206a53b782b371cfe2d02da932c8c173.png'
$html = defaultLinkTo($html, static::URI);
$articles = $html->find('div.crayons-story')
or returnServerError('Could not find articles!');
or throwServerException('Could not find articles!');
foreach ($articles as $article) {
$item = [];

View File

@@ -204,13 +204,13 @@ class Drive2ruBridge extends BridgeAbstract
break;
case 'Бортжурналы (По модели или марке)':
if (!preg_match('/^https:\/\/www.drive2.ru\/experience/', $this->getInput('url'))) {
returnServerError('Invalid url');
throwServerException('Invalid url');
}
$this->getLogbooksContent($this->getInput('url'));
break;
case 'Личные блоги':
if (!preg_match('/^[a-zA-Z0-9-]{3,16}$/', $this->getInput('username'))) {
returnServerError('Invalid username');
throwServerException('Invalid username');
}
$this->getUserContent('https://www.drive2.ru/users/' . $this->getInput('username'));
break;

View File

@@ -88,12 +88,9 @@ class EdfPricesBridge extends BridgeAbstract
$tablePrices = $html
->find('#grille-tarifaire-et-prix-du-kwh-du-tarif-reglemente-edf-en-option-base', 0)
->nextSibling()
->nextSibling()
->nextSibling();
$prices = $tablePrices->find('.table--stripped tbody tr');
// last element is useless because part of another table
array_pop($prices);
$prices = $tablePrices->find('.table tbody tr');
// price per kWh is same for all powers
if ($prices && count($prices) === 9) {
@@ -122,12 +119,9 @@ class EdfPricesBridge extends BridgeAbstract
$tablePrices = $html
->find('#grille-tarifaire-et-prix-du-kwh-du-tarif-reglemente-edf-en-option-heures-pleines-heures-creuses', 0)
->nextSibling()
->nextSibling()
->nextSibling();
$prices = $tablePrices->find('.table--stripped tbody tr');
// last element is useless because part of another table
array_pop($prices);
$prices = $tablePrices->find('.table tbody tr');
// price per kWh is same for all powers
if ($prices && count($prices) === 8) {
@@ -158,14 +152,14 @@ class EdfPricesBridge extends BridgeAbstract
private function ejp(simple_html_dom $html, string $contractUri, int $power): void
{
$tablePrices = $html
->find('#grille-tarifaire-et-prix-du-kwh-du-tarif-reglemente-edf-en-option-ejp', 0)
->find('#ejp', 0)
->nextSibling()
->nextSibling()
->nextSibling()
->nextSibling()
->nextSibling();
$prices = $tablePrices->find('.table--stripped tbody tr');
// last element is useless because part of another table
array_pop($prices);
$prices = $tablePrices->find('.table tbody tr');
// price per kWh is same for all powers
if ($prices && count($prices) === 5) {
@@ -190,13 +184,11 @@ class EdfPricesBridge extends BridgeAbstract
private function addSubscriptionPowerInfo(simple_html_dom_node $tablePrices, string $contractUri, int $power, int $numberOfPrices): void
{
$prices = $tablePrices->find('.table--stripped tbody tr');
// last element is useless because part of another table
array_pop($prices);
$prices = $tablePrices->find('.table tbody tr');
// 7 contracts for tempo: 6, 9, 12, 15, 18, 30 and 36 kVA
// 8 contracts for tempo: 6, 9, 12, 15, 18, 24, 30 and 36 kVA
// 9 contracts for base: 3, 6, 9, 12, 15, 18, 24, 30 and 36 kVA
// 7 contracts for HPHC: 6, 9, 12, 15, 18, 24, 30 and 36 kVA
// 8 contracts for HPHC: 6, 9, 12, 15, 18, 24, 30 and 36 kVA
// 5 contracts for EJP: 9, 12, 15, 18 and 36 kVA
if ($prices && count($prices) === $numberOfPrices) {
$powerFound = false;

View File

@@ -160,7 +160,7 @@ class ElektroARGOSBridge extends BridgeAbstract
{
// Check if page contains articles and split by class
$articles = $html->find('.com-news-feature-prerex') or
returnServerError('No articles found! Layout might have changed!');
throwServerException('No articles found! Layout might have changed!');
// Articles loop
foreach ($articles as $article) {
@@ -189,7 +189,7 @@ class ElektroARGOSBridge extends BridgeAbstract
{
// Check if page contains articles and split by class
$articles = $html->find('.com-news-common-prerex') or
returnServerError('No articles found! Layout might have changed!');
throwServerException('No articles found! Layout might have changed!');
// Articles loop
foreach ($articles as $article) {
@@ -225,7 +225,7 @@ class ElektroARGOSBridge extends BridgeAbstract
{
// Check if page contains articles and split by class
$articles = $html->find('.com-news-common-prerex') or
returnServerError('No articles found! Layout might have changed!');
throwServerException('No articles found! Layout might have changed!');
// Articles loop
foreach ($articles as $article) {
@@ -273,7 +273,7 @@ class ElektroARGOSBridge extends BridgeAbstract
{
// Return URI of the article
$element = $article->find('a', 0) or
returnServerError('Anchor not found!');
throwServerException('Anchor not found!');
return $element->href;
}
@@ -307,7 +307,7 @@ class ElektroARGOSBridge extends BridgeAbstract
{
// Check if date is set
$element = $article->find('div.com-news-common-prerex__date', 0) or
returnServerError('Date not found!');
throwServerException('Date not found!');
return $element->plaintext;
}
@@ -322,7 +322,7 @@ class ElektroARGOSBridge extends BridgeAbstract
{
// Extract description
$element = $article->find('ul.ws-product-information__piece-description', 0)->find('li', 0) or
returnServerError('Description not found!');
throwServerException('Description not found!');
return $element->innertext;
}
@@ -337,7 +337,7 @@ class ElektroARGOSBridge extends BridgeAbstract
{
// Extract description
$element = $article->find('div.ws-product-price-validity', 0)->find('div', 0) or
returnServerError('Description not found!');
throwServerException('Description not found!');
return $element->innertext;
}
@@ -352,7 +352,7 @@ class ElektroARGOSBridge extends BridgeAbstract
{
// Extract description
$element = $article->find('div.ws-product-price-validity', 0)->find('div', 1) or
returnServerError('Description not found!');
throwServerException('Description not found!');
return $element->innertext;
}
@@ -454,7 +454,7 @@ class ElektroARGOSBridge extends BridgeAbstract
{
// Extract title
$element = $article->find('img', 0) or
returnServerError('Title not found!');
throwServerException('Title not found!');
return $element->alt;
}
@@ -469,7 +469,7 @@ class ElektroARGOSBridge extends BridgeAbstract
{
// Extract title
$element = $article->find('div.com-news-common-prerex__right-box', 0)->find('h3', 0)
or returnServerError('Title not found!');
or throwServerException('Title not found!');
return $element->plaintext;
}

View File

@@ -59,13 +59,20 @@ class EpicGamesFreeBridge extends BridgeAbstract
) {
continue;
}
$slug = $element['catalogNs']['mappings'][0]['pageSlug'] ?? null;
if ($slug !== null) {
$uri = parent::getURI() . $this->getInput('locale') . '/p/' . $slug;
} else {
// slug not found, show the root promos page
$uri = parent::getURI() . $this->getInput('locale') . '/free-games';
}
$item = [
'author' => $element['seller']['name'],
'content' => $element['description'],
'enclosures' => array_map(fn($item) => $item['url'], $element['keyImages']),
'timestamp' => strtotime($promo['startDate']),
'title' => $element['title'],
'uri' => parent::getURI() . $this->getInput('locale') . '/p/' . $element['productSlug'],
'uri' => $uri,
];
$this->items[] = $item;
}

View File

@@ -36,6 +36,9 @@ class ExplosmBridge extends BridgeAbstract
$html = getSimpleHTMLDOM($url);
$element = $html->find('[class*=ComicImage]', 0);
if (!$element) {
break; // skip, if element was not found
}
$date = $element->find('[class^=Author__Right] p', 0)->plaintext;
$author = str_replace('by ', '', $element->find('[class^=Author__Right] p', 1)->plaintext);
$image = $element->find('img', 0)->src;

View File

@@ -85,13 +85,13 @@ class FB2Bridge extends BridgeAbstract
$pageInfo = $this->getPageInfos($page, $cookies);
if ($pageInfo['userId'] === null) {
returnClientError(
throwClientException(
<<<EOD
Unable to get the page id. You should consider getting the ID by hand, then importing it into FB2Bridge
EOD
);
} elseif ($pageInfo['userId'] == -1) {
returnClientError(
throwClientException(
<<<EOD
This page is not accessible without being logged in.
EOD

43
bridges/FabBridge.php Normal file
View File

@@ -0,0 +1,43 @@
<?php
class FabBridge extends BridgeAbstract
{
const NAME = 'Epic Games Fab.com';
const URI = 'https://www.fab.com';
const DESCRIPTION = 'Limited-Time Free Game Engine Assets';
const MAINTAINER = 'thefranke';
const CACHE_TIMEOUT = 86400;
public function collectData()
{
$url = static::URI . '/i/listings/search?is_discounted=1&is_free=1';
$header = [
'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:139.0) Gecko/20100101 Firefox/139.0',
'Accept: application/json, text/plain, */*',
'Accept-Language: en',
'Accept-Encoding: gzip, deflate, br, zstd',
'Referer: ' . static::URI
];
$json = getContents($url, $header);
$json = json_decode($json);
foreach ($json->results as $item) {
$thumbnail = $item->thumbnails[0]->mediaUrl;
$itemurl = static::URI . '/listings/' . $item->uid;
$itemapiurl = static::URI . '/i/listings/' . $item->uid;
$itemjson = getContents($itemapiurl, $header);
$itemjson = json_decode($itemjson);
$this->items[] = [
'title' => $item->title,
'author' => $item->user->sellerName,
'uri' => $itemurl,
'timestamp' => strtotime($item->lastUpdatedAt),
'content' => '<a href="' . $itemurl . '"><img src="' . $thumbnail . '"></a>' . $itemjson->description,
];
}
}
}

View File

@@ -155,7 +155,7 @@ class FacebookBridge extends BridgeAbstract
break;
default:
returnClientError('Unknown context: "' . $this->queriedContext . '"!');
throwClientException('Unknown context: "' . $this->queriedContext . '"!');
}
$limit = $this->getInput('limit') ?: -1;
@@ -184,7 +184,7 @@ class FacebookBridge extends BridgeAbstract
$html = getSimpleHTMLDOM($touchURI, $header);
if (!$this->isPublicGroup($html)) {
returnClientError('This group is not public! RSS-Bridge only supports public groups!');
throwClientException('This group is not public! RSS-Bridge only supports public groups!');
}
defaultLinkTo($html, substr(self::URI, 0, strlen(self::URI) - 1));
@@ -192,7 +192,7 @@ class FacebookBridge extends BridgeAbstract
$this->groupName = $this->extractGroupName($html);
$posts = $html->find('div.story_body_container')
or returnServerError('Failed finding posts!');
or throwServerException('Failed finding posts!');
foreach ($posts as $post) {
$item = [];
@@ -224,7 +224,7 @@ class FacebookBridge extends BridgeAbstract
return explode('/', $urlparts['path'])[2];
} elseif (strpos($group, '/') !== false) {
returnClientError('The group you provided is invalid: ' . $group);
throwClientException('The group you provided is invalid: ' . $group);
} else {
return $group;
}
@@ -246,7 +246,7 @@ class FacebookBridge extends BridgeAbstract
$provided_host !== $facebook_host
&& 'www.' . $provided_host !== $facebook_host
) {
returnClientError('The host you provided is invalid! Received "'
throwClientException('The host you provided is invalid! Received "'
. $provided_host
. '", expected "'
. $facebook_host
@@ -268,7 +268,7 @@ class FacebookBridge extends BridgeAbstract
private function extractGroupName($html)
{
$ogtitle = $html->find('._de1', 0)
or returnServerError('Unable to find group title!');
or throwServerException('Unable to find group title!');
return html_entity_decode($ogtitle->plaintext, ENT_QUOTES);
}
@@ -276,7 +276,7 @@ class FacebookBridge extends BridgeAbstract
private function extractGroupPostURI($post)
{
$elements = $post->find('a')
or returnServerError('Unable to find URI!');
or throwServerException('Unable to find URI!');
foreach ($elements as $anchor) {
// Find the one that is a permalink
@@ -292,7 +292,7 @@ class FacebookBridge extends BridgeAbstract
private function extractGroupPostContent($post)
{
$content = $post->find('div._5rgt', 0)
or returnServerError('Unable to find user content!');
or throwServerException('Unable to find user content!');
$context_text = $content->innertext;
if ($content->next_sibling() !== null) {
@@ -304,7 +304,7 @@ class FacebookBridge extends BridgeAbstract
private function extractGroupPostAuthor($post)
{
$element = $post->find('h3 a', 0)
or returnServerError('Unable to find author information!');
or throwServerException('Unable to find author information!');
return $element->plaintext;
}
@@ -334,7 +334,7 @@ class FacebookBridge extends BridgeAbstract
private function extractGroupPostTitle($post)
{
$element = $post->find('h3', 0)
or returnServerError('Unable to find title!');
or throwServerException('Unable to find title!');
if (strpos($element->plaintext, 'shared') === false) {
$content = strip_tags($this->extractGroupPostContent($post));
@@ -370,14 +370,14 @@ class FacebookBridge extends BridgeAbstract
!array_key_exists('path', $urlparts)
|| $urlparts['path'] === '/'
) {
returnClientError('The URL you provided doesn\'t contain the user name!');
throwClientException('The URL you provided doesn\'t contain the user name!');
}
return explode('/', $urlparts['path'])[1];
} else {
// First character cannot be a forward slash
if (strpos($user, '/') === 0) {
returnClientError('Remove leading slash "/" from the username!');
throwClientException('Remove leading slash "/" from the username!');
}
return $user;
@@ -572,7 +572,7 @@ EOD;
$loginForm = $html->find('._585r', 0);
if ($loginForm != null) {
returnServerError('You must be logged in to view this page. This is not supported by RSS-Bridge.');
throwServerException('You must be logged in to view this page. This is not supported by RSS-Bridge.');
}
$mainColumn = $html->find('#pagelet_timeline_main_column');

View File

@@ -37,13 +37,14 @@ class FallGuysBridge extends BridgeAbstract
public function collectData()
{
$html = getSimpleHTMLDOM(self::getURI());
$newsData = self::requestJsonData(self::getURI(), false);
$data = json_decode($html->find('#__NEXT_DATA__', 0)->innertext);
foreach ($newsData->props->pageProps->newsList as $newsItem) {
$newsItemUrl = self::getURI() . '/' . $newsItem->slug;
$newsItemTitle = $newsItem->header->title;
foreach ($data->props->pageProps->newsList as $newsItem) {
$headerDescription = property_exists($newsItem->header, 'description') ? $newsItem->header->description : '';
$headerImage = $newsItem->header->image->src;
$headerImage = $newsItem->newsLandingConfig->options[0]->image->src->url;
$contentImages = [$headerImage];
@@ -52,67 +53,79 @@ class FallGuysBridge extends BridgeAbstract
<p><img src="{$headerImage}"></p>
HTML;
foreach ($newsItem->content->items as $contentItem) {
if (property_exists($contentItem, 'articleCopy')) {
if (property_exists($contentItem->articleCopy, 'title')) {
$title = $contentItem->articleCopy->title;
try {
$newsItemData = self::requestJsonData($newsItemUrl, true);
} catch (\Exception $e) {
$this->logger->error(sprintf('Failed to request data for news item "%s" (%s)', $newsItemTitle, $newsItemUrl), ['e' => $e]);
$newsItemData = null;
}
if (!$newsItemData) {
$this->logger->error(sprintf('Failed to parse json data for news item "%s" (%s)', $newsItemTitle, $newsItemUrl));
} else {
foreach ($newsItemData->props->pageProps->pageData->content->items as $contentItem) {
if (property_exists($contentItem, 'articleCopy')) {
if (property_exists($contentItem->articleCopy, 'title')) {
$title = $contentItem->articleCopy->title;
$content .= <<<HTML
<h2>{$title}</h2>
HTML;
}
$text = $contentItem->articleCopy->copy;
$content .= <<<HTML
<h2>{$title}</h2>
<p>{$text}</p>
HTML;
}
} elseif (property_exists($contentItem, 'articleImage')) {
$image = $contentItem->articleImage->imageSrc;
$text = $contentItem->articleCopy->copy;
if ($image != $headerImage) {
$contentImages[] = $image;
$content .= <<<HTML
<p>{$text}</p>
HTML;
} elseif (property_exists($contentItem, 'articleImage')) {
$image = $contentItem->articleImage->imageSrc;
$content .= <<<HTML
<p><img src="{$image}"></p>
HTML;
}
} elseif (property_exists($contentItem, 'embeddedVideo')) {
$mediaOptions = $contentItem->embeddedVideo->mediaOptions;
$mainContentOptions = $contentItem->embeddedVideo->mainContentOptions;
if ($image != $headerImage) {
$contentImages[] = $image;
if (count($mediaOptions) == count($mainContentOptions)) {
for ($i = 0; $i < count($mediaOptions); $i++) {
if (property_exists($mediaOptions[$i], 'youtubeVideo')) {
$videoUrl = 'https://youtu.be/' . $mediaOptions[$i]->youtubeVideo->contentId;
$image = $mainContentOptions[$i]->image->src ?? '';
$content .= <<<HTML
<p><img src="{$image}"></p>
HTML;
}
} elseif (property_exists($contentItem, 'embeddedVideo')) {
$mediaOptions = $contentItem->embeddedVideo->mediaOptions;
$mainContentOptions = $contentItem->embeddedVideo->mainContentOptions;
$content .= '<p>';
if (count($mediaOptions) == count($mainContentOptions)) {
for ($i = 0; $i < count($mediaOptions); $i++) {
if (property_exists($mediaOptions[$i], 'youtubeVideo')) {
$videoUrl = 'https://youtu.be/' . $mediaOptions[$i]->youtubeVideo->contentId;
$image = $mainContentOptions[$i]->image->src ?? '';
if ($image != $headerImage) {
$contentImages[] = $image;
$content .= '<p>';
if ($image != $headerImage) {
$contentImages[] = $image;
$content .= <<<HTML
<a href="{$videoUrl}"><img src="{$image}"></a><br>
HTML;
}
$content .= <<<HTML
<a href="{$videoUrl}"><img src="{$image}"></a><br>
<i>(Video: <a href="{$videoUrl}">{$videoUrl}</a>)</i>
HTML;
$content .= '</p>';
}
$content .= <<<HTML
<i>(Video: <a href="{$videoUrl}">{$videoUrl}</a>)</i>
HTML;
$content .= '</p>';
}
}
} else {
$this->logger->warning(sprintf('Unsupported content item in news item "%s" (%s)', $newsItemTitle, $newsItemUrl));
}
}
}
$item = [
'uid' => $newsItem->_id,
'uri' => self::getURI() . '/' . $newsItem->_slug,
'title' => $newsItem->_title,
'timestamp' => $newsItem->lastModified,
'uid' => $newsItem->id,
'uri' => $newsItemUrl,
'title' => $newsItemTitle,
'timestamp' => $newsItem->activeDate,
'content' => $content,
'enclosures' => $contentImages,
];
@@ -131,4 +144,12 @@ class FallGuysBridge extends BridgeAbstract
{
return self::BASE_URI . '/favicon.ico';
}
private function requestJsonData(string $url, bool $useCache)
{
$html = $useCache ? getSimpleHTMLDOMCached($url) : getSimpleHTMLDOM($url);
$jsonElement = $html->find('#__NEXT_DATA__', 0);
$json = $jsonElement ? $jsonElement->innertext : null;
return json_decode($json);
}
}

View File

@@ -0,0 +1,95 @@
<?php
class FanaticalBridge extends BridgeAbstract
{
const NAME = 'Fanatical';
const MAINTAINER = 'phantop';
const URI = 'https://www.fanatical.com/en/';
const DESCRIPTION = 'Returns bundles from Fanatical.';
const PARAMETERS = [[
'type' => [
'name' => 'Bundle type',
'type' => 'list',
'defaultValue' => 'all',
'values' => [
'All' => 'all',
'Books' => 'book-',
'ELearning' => 'elearning-',
'Games' => '',
'Software' => 'software-',
]
]
]];
const IMGURL = 'https://fanatical.imgix.net/product/original/';
public function collectData()
{
$api = 'https://www.fanatical.com/api/all/en';
$json = json_decode(getContents($api), true)['pickandmix'];
$type = $this->getInput('type');
foreach ($json as $element) {
if ($type != 'all') {
if ($element['type'] != $type . 'bundle') {
continue;
}
}
$item = [
'categories' => [$element['type']],
'content' => '<ul>',
'enclosures' => [self::IMGURL . $element['cover_image']],
'timestamp' => $element['valid_from'],
'title' => $element['name'],
'uri' => parent::getURI() . 'pick-and-mix/' . $element['slug'],
];
$slugs = [];
foreach ($element['products'] as $product) {
$slug = $product['slug'];
if (in_array($slug, $slugs)) {
continue;
}
$slugs[] = $slug;
$uri = parent::getURI() . 'game/' . $slug;
$item['content'] .= '<li><a href="' . $uri . '">' . $product['name'] . '</a></li>';
$item['enclosures'][] = self::IMGURL . $product['cover'];
}
foreach ($element['tiers'] as $tier) {
$count = $tier['quantity'];
$price = round($tier['price']['USD'] / 100, 2);
$per = round($price / $count, 2);
$item['categories'][] = "$count at $per for $price total";
}
$item['content'] .= '</ul>';
$this->items[] = $item;
}
}
public function getName()
{
$name = parent::getName();
$name .= $this->getKey('type') ? ' - ' . $this->getKey('type') : '';
return $name;
}
public function getURI()
{
$uri = parent::getURI();
$type = $this->getKey('type');
if ($type) {
$uri .= 'bundle/';
if ($type != 'All') {
$uri .= strtolower($type);
}
}
return $uri;
}
public function getIcon()
{
return 'https://cdn.fanatical.com/production/icons/fanatical-icon-android-chrome-192x192.png';
}
}

View File

@@ -40,7 +40,7 @@ class FeedExpanderExampleBridge extends FeedExpander
parent::collectExpandableDatas('http://segfault.linuxmint.com/feed/atom/');
break;
default:
returnClientError('Unknown version ' . $this->getInput('version') . '!');
throwClientException('Unknown version ' . $this->getInput('version') . '!');
}
}
}

View File

@@ -58,13 +58,13 @@ class FirefoxAddonsBridge extends BridgeAbstract
}
$item['content'] = <<<EOD
<strong>Release Notes</strong>
<p><strong>Release Notes</strong></p>
<p>{$releaseNotes}</p>
<strong>Compatibility</strong>
<p><strong>Compatibility</strong></p>
<p>{$compatibility}</p>
<strong>License</strong>
<p><strong>License</strong></p>
<p>{$license}</p>
<strong>Download</strong>
<p><strong>Download</strong></p>
<p><a href="{$downloadlink}">{$xpiFilename}</a> ($size)</p>
EOD;

View File

@@ -1,52 +0,0 @@
<?php
class FirstLookMediaTechBridge extends BridgeAbstract
{
const NAME = 'First Look Media - Technology';
const URI = 'https://tech.firstlook.media';
const DESCRIPTION = 'First Look Media Technology page';
const MAINTAINER = 'somini';
const PARAMETERS = [
[
'projects' => [
'type' => 'checkbox',
'name' => 'Include Projects?',
]
]
];
public function collectData()
{
$html = getSimpleHTMLDOM(self::URI);
if ($this->getInput('projects')) {
$top_projects = $html->find('.PromoList-ul', 0);
foreach ($top_projects->find('li.PromoList-item') as $element) {
$item = [];
$item_uri = $element->find('a', 0);
$item['uri'] = $item_uri->href;
$item['title'] = strip_tags($item_uri->innertext);
$item['content'] = $element->find('div > div', 0);
$this->items[] = $item;
}
}
$top_articles = $html->find('.PromoList-ul', 1);
foreach ($top_articles->find('li.PromoList-item') as $element) {
$item = [];
$item_left = $element->find('div > div', 0);
$item_date = $element->find('.PromoList-date', 0);
$item['timestamp'] = strtotime($item_date->innertext);
$item_date->outertext = ''; /* Remove */
$item['author'] = $item_left->innertext;
$item_uri = $element->find('a', 0);
$item['uri'] = self::URI . $item_uri->href;
$item['title'] = strip_tags($item_uri);
$this->items[] = $item;
}
}
}

View File

@@ -112,7 +112,7 @@ class FlickrBridge extends BridgeAbstract
break;
default:
returnClientError('Invalid context: ' . $this->queriedContext);
throwClientException('Invalid context: ' . $this->queriedContext);
}
$model_json = $this->extractJsonModel($html);

View File

@@ -42,7 +42,7 @@ class Formula1Bridge extends BridgeAbstract
'locale: en'
]));
if (property_exists($json, 'error')) {
returnServerError($json->message);
throwServerException($json->message);
}
$list = $json->items;

View File

@@ -40,7 +40,7 @@ class FunkBridge extends BridgeAbstract
}
break;
default:
returnServerError('Unknown context!');
throwServerException('Unknown context!');
}
}

View File

@@ -9,20 +9,19 @@ class GOGBridge extends BridgeAbstract
public function collectData()
{
$values = getContents('https://www.gog.com/games/ajax/filtered?limit=25&sort=new');
$values = getContents('https://catalog.gog.com/v1/catalog?limit=48&order=desc%3AstoreReleaseDate');
$decodedValues = json_decode($values);
$limit = 0;
foreach ($decodedValues->products as $game) {
$item = [];
$item['author'] = $game->developer . ' / ' . $game->publisher;
$item['author'] = implode(', ', $game->developers) . ' / ' . implode(', ', $game->publishers);
$item['title'] = $game->title;
$item['id'] = $game->id;
$item['uri'] = self::URI . $game->url;
$item['uri'] = $game->storeLink;
$item['content'] = $this->buildGameContentPage($game);
$item['timestamp'] = $game->globalReleaseDate;
foreach ($game->gallery as $image) {
foreach ($game->screenshots as $image) {
$item['enclosures'][] = $image . '.jpg';
}
@@ -42,18 +41,10 @@ class GOGBridge extends BridgeAbstract
$gameDescriptionValue = json_decode($gameDescriptionText);
$content = 'Genres: ';
$content .= implode(', ', $game->genres);
$content .= implode(', ', array_column($game->genres, 'name'));
$content .= '<br />Supported Platforms: ';
if ($game->worksOn->Windows) {
$content .= 'Windows ';
}
if ($game->worksOn->Mac) {
$content .= 'Mac ';
}
if ($game->worksOn->Linux) {
$content .= 'Linux ';
}
$content .= implode(', ', $game->operatingSystems);
$content .= '<br />' . $gameDescriptionValue->description->full;

View File

@@ -56,7 +56,7 @@ class GitHubGistBridge extends BridgeAbstract
$html = defaultLinkTo($html, $this->getURI());
$fileinfo = $html->find('[class~="file-info"]', 0)
or returnServerError('Could not find file info!');
or throwServerException('Could not find file info!');
$this->filename = $fileinfo->plaintext;
@@ -68,18 +68,18 @@ class GitHubGistBridge extends BridgeAbstract
foreach ($comments as $comment) {
$uri = $comment->find('a[href*=#gistcomment]', 0)
or returnServerError('Could not find comment anchor!');
or throwServerException('Could not find comment anchor!');
$title = $comment->find('h3', 0);
$datetime = $comment->find('[datetime]', 0)
or returnServerError('Could not find comment datetime!');
or throwServerException('Could not find comment datetime!');
$author = $comment->find('a.author', 0)
or returnServerError('Could not find author name!');
or throwServerException('Could not find author name!');
$message = $comment->find('[class~="comment-body"]', 0)
or returnServerError('Could not find comment body!');
or throwServerException('Could not find comment body!');
$item = [];

View File

@@ -188,7 +188,7 @@ class GiteaBridge extends BridgeAbstract
protected function collectReleasesData($html)
{
$releases = $html->find('#release-list > li')
or returnServerError('Unable to find releases');
or throwServerException('Unable to find releases');
foreach ($releases as $release) {
$this->items[] = [
@@ -203,7 +203,7 @@ class GiteaBridge extends BridgeAbstract
protected function collectTagsData($html)
{
$tags = $html->find('table#tags-table > tbody > tr')
or returnServerError('Unable to find tags');
or throwServerException('Unable to find tags');
foreach ($tags as $tag) {
$this->items[] = [
@@ -216,7 +216,7 @@ class GiteaBridge extends BridgeAbstract
protected function collectCommitsData($html)
{
$commits = $html->find('#commits-table tbody tr')
or returnServerError('Unable to find commits');
or throwServerException('Unable to find commits');
foreach ($commits as $commit) {
$this->items[] = [
@@ -232,7 +232,7 @@ class GiteaBridge extends BridgeAbstract
protected function collectIssuesData($html)
{
$issues = $html->find('.issue.list li')
or returnServerError('Unable to find issues');
or throwServerException('Unable to find issues');
foreach ($issues as $issue) {
$uri = $issue->find('a', 0)->href;
@@ -259,7 +259,7 @@ class GiteaBridge extends BridgeAbstract
protected function collectSingleIssueOrPrData($html)
{
$comments = $html->find('.comment')
or returnServerError('Unable to find comments');
or throwServerException('Unable to find comments');
foreach ($comments as $comment) {
if (
@@ -293,7 +293,7 @@ class GiteaBridge extends BridgeAbstract
protected function collectPullRequestsData($html)
{
$issues = $html->find('.issue.list li')
or returnServerError('Unable to find pull requests');
or throwServerException('Unable to find pull requests');
foreach ($issues as $issue) {
$uri = $issue->find('a', 0)->href;

View File

@@ -98,7 +98,7 @@ class GlassdoorBridge extends BridgeAbstract
private function collectBlogData($html, $limit)
{
$posts = $html->find('div.post')
or returnServerError('Unable to find blog posts!');
or throwServerException('Unable to find blog posts!');
foreach ($posts as $post) {
$item = [];
@@ -121,7 +121,7 @@ class GlassdoorBridge extends BridgeAbstract
private function collectReviewData($html, $limit)
{
$reviews = $html->find('#ReviewsFeed li[id^="empReview]')
or returnServerError('Unable to find reviews!');
or throwServerException('Unable to find reviews!');
foreach ($reviews as $review) {
$item = [];
@@ -163,7 +163,7 @@ class GlassdoorBridge extends BridgeAbstract
FILTER_FLAG_PATH_REQUIRED
)
) {
returnClientError('The specified URL is invalid!');
throwClientException('The specified URL is invalid!');
}
$uri = filter_var($uri, FILTER_SANITIZE_URL);
@@ -189,7 +189,7 @@ class GlassdoorBridge extends BridgeAbstract
];
if (!in_array($parts[1], $allowed_strings)) {
returnClientError('Please specify a URL pointing to the companies review page!');
throwClientException('Please specify a URL pointing to the companies review page!');
}
return $uri;

View File

@@ -31,25 +31,33 @@ class GoComicsBridge extends BridgeAbstract
public function collectData()
{
$link = $this->getURI();
$landingpage = getSimpleHTMLDOM($link);
$element = $landingpage->find('div[data-post-url]', 0);
if ($element) {
$link = $element->getAttribute('data-post-url');
} else { // fallback for comics without data-post-url (assumes daily comic)
$nextcomiclink = $landingpage->find('a[class*="ComicNavigation_controls__button_previous__"]', 0)->href;
preg_match('/(\d{4}\/\d{2}\/\d{2})/', $nextcomiclink, $nclmatches);
if (!empty($nclmatches[1])) {
$nextdate = new DateTime($nclmatches[1]);
$nextdate = $nextdate->modify('+1 day')->format('Y/m/d');
$link = $link . '/' . $nextdate;
} else {
throw new \Exception('Could not find the first comic URL. Please create a new GitHub issue.');
}
}
for ($i = 0; $i < $this->getInput('limit'); $i++) {
$html = getSimpleHTMLDOM($link);
// get json data from the first page
$json = $html->find('div[class^="ShowComicViewer_showComicViewer__comic__"] script[type="application/ld+json"]', 0)->innertext;
$data = json_decode($json, false);
$html = getSimpleHTMLDOMCached($link, 86400);
$imagelink = $html->find('meta[property="og:image"]', 0)->content;
$parts = explode('/', $link);
$date = DateTime::createFromFormat('Y/m/d', implode('/', array_slice($parts, -3)));
$title = $html->find('meta[property="og:title"]', 0)->content;
preg_match('/by (.*?) for/', $title, $authormatches);
$author = $authormatches[1] ?? 'GoComics';
$item = [];
$author = $data->author->name;
$imagelink = $data->contentUrl;
$date = $data->datePublished;
$title = $data->name . ' - GoComics';
// get a permlink for this day's comic if there isn't one specified
if ($link === $this->getURI()) {
$link = $this->getURI() . '/' . DateTime::createFromFormat('F j, Y', $date)->format('Y/m/d');
}
$item['id'] = $imagelink;
$item['uri'] = $link;
$item['author'] = $author;
@@ -57,7 +65,7 @@ class GoComicsBridge extends BridgeAbstract
if ($this->getInput('date-in-title') === true) {
$item['title'] = $title;
}
$item['timestamp'] = DateTime::createFromFormat('F j, Y', $date)->setTime(0, 0, 0)->getTimestamp();
$item['timestamp'] = $date->setTime(0, 0, 0)->getTimestamp();
$item['content'] = '<img src="' . $imagelink . '" />';
$link = rtrim(self::URI, '/') . $html->find('a[class*="ComicNavigation_controls__button_previous__"]', 0)->href;

View File

@@ -141,7 +141,7 @@ class GogsBridge extends BridgeAbstract
protected function collectCommitsData($html)
{
$commits = $html->find('#commits-table tbody tr')
or returnServerError('Unable to find commits');
or throwServerException('Unable to find commits');
foreach ($commits as $commit) {
$this->items[] = [
@@ -157,7 +157,7 @@ class GogsBridge extends BridgeAbstract
protected function collectIssuesData($html)
{
$issues = $html->find('.issue.list li')
or returnServerError('Unable to find issues');
or throwServerException('Unable to find issues');
foreach ($issues as $issue) {
$uri = $issue->find('a', 0)->href;
@@ -185,7 +185,7 @@ class GogsBridge extends BridgeAbstract
protected function collectSingleIssueData($html)
{
$comments = $html->find('.comments .comment')
or returnServerError('Unable to find comments');
or throwServerException('Unable to find comments');
foreach ($comments as $comment) {
$this->items[] = [
@@ -203,7 +203,7 @@ class GogsBridge extends BridgeAbstract
protected function collectReleasesData($html)
{
$releases = $html->find('#release-list li')
or returnServerError('Unable to find releases');
or throwServerException('Unable to find releases');
foreach ($releases as $release) {
$this->items[] = [

View File

@@ -132,13 +132,22 @@ class GolemBridge extends FeedExpander
// delete known bad elements
foreach (
$article->find('div[id*="adtile"], #job-market, #seminars, iframe,
div.gbox_affiliate, div.toc') as $bad
.gbox_affiliate, div.toc') as $bad
) {
$bad->remove();
}
// reload html, as remove() is buggy
$article = str_get_html($article->outertext);
// Add multipage headers, but only if they are different to the article header
$firstHeader = $page->find('.table-jtoc td', 0);
if (isset($firstHeader)) {
$firstHeader = html_entity_decode($firstHeader->title);
}
$multipageHeader = $article->find('header.paged-cluster-header h1', 0);
if (isset($multipageHeader) && $multipageHeader->plaintext !== $firstHeader) {
$item .= $multipageHeader;
}
$header = $article->find('header', 0);
foreach ($header->find('p, figure') as $element) {
@@ -152,7 +161,7 @@ class GolemBridge extends FeedExpander
$img->src = $img->getAttribute('data-src-full');
}
foreach ($content->find('p, h1, h2, h3, pre, img[src*="."], iframe, video') as $element) {
foreach ($content->find('p, h1, h2, h3, pre, img[src*="."], div[class*="golem_tablediv"], iframe, video') as $element) {
$item .= $element;
}

View File

@@ -26,7 +26,7 @@ class GoogleSearchBridge extends BridgeAbstract
// todo: wrap this in try..catch because 429 too many requests happens a lot
$dom = getSimpleHTMLDOM($this->getURI(), ['Accept-language: en-US']);
if (!$dom) {
returnServerError('No results for this query.');
throwServerException('No results for this query.');
}
$result = $dom->find('div[id=res]', 0);

View File

@@ -48,7 +48,7 @@ class HaveIBeenPwnedBridge extends BridgeAbstract
. $pwnCount . ' breached accounts';
$item['dateAdded'] = $breach['AddedDate'];
$item['breachDate'] = $breach['BreachDate'];
$item['uri'] = self::URI . '/PwnedWebsites#' . $breach['Name'];
$item['uri'] = self::URI . '/breach/' . $breach['Name'];
$item['content'] = '<p>' . $breach['Description'] . '</p>';
$item['content'] .= '<p>' . $this->breachType($breach) . '</p>';

View File

@@ -138,6 +138,7 @@ class HeiseBridge extends FeedExpander
}
// abort on heise+ articles
if ($sessioncookie == '' && str_starts_with($item['title'], 'heise+ |')) {
$item['uri'] = 'https://archive.is/' . $item['uri'];
return $item;
}
@@ -162,7 +163,7 @@ class HeiseBridge extends FeedExpander
// remove unwanted stuff
foreach (
$article->find('figure.branding, figure.a-inline-image, a-ad, div.ho-text, a-img,
.a-toc__list, a-collapse, .opt-in__description, .opt-in__footnote') as $element
.a-toc__list, a-collapse, .opt-in__description, .opt-in__footnote, .notice-banner__text, .notice-banner__link, .ad, .ad--inread') as $element
) {
$element->remove();
}

View File

@@ -34,25 +34,90 @@ class HumbleBundleBridge extends BridgeAbstract
}
foreach ($products as $element) {
$item = [];
$item['author'] = $element['author'];
$item['timestamp'] = $element['start_date|datetime'];
$item['title'] = $element['tile_short_name'];
$item['uid'] = $element['machine_name'];
$item['uri'] = parent::getURI() . $element['product_url'];
$dom = new simple_html_dom();
$body = $dom->createElement('div');
$item = [
'author' => $element['author'],
'categories' => $element['hover_highlights'],
'content' => $body,
'timestamp' => $element['start_date|datetime'],
'title' => $element['tile_short_name'],
'uid' => $element['machine_name'],
'uri' => parent::getURI() . $element['product_url'],
];
$item['content'] = $element['marketing_blurb'];
$item['content'] .= '<br>' . $element['detailed_marketing_blurb'];
$item['categories'] = $element['hover_highlights'];
array_unshift($item['categories'], explode(':', $element['tile_name'])[0]);
array_unshift($item['categories'], $element['tile_stamp']);
$item['enclosures'] = [$element['tile_logo'], $element['high_res_tile_image']];
$this->items[] = $item;
$this->createChild($dom, $body, 'img', null, ['src' => $element['tile_logo']]);
$this->createChild($dom, $body, 'img', null, ['src' => $element['high_res_tile_image']]);
$this->createChild($dom, $body, 'h2', $element['short_marketing_blurb']);
$this->createChild($dom, $body, 'p', $element['detailed_marketing_blurb']);
$this->items[] = $this->processBundle($item, $dom, $body);
}
}
private function createChild($dom, $body, $name = null, $val = null, $args = [])
{
if ($name == null) {
$elem = $dom->createTextNode($val);
} else {
$elem = $dom->createElement($name, $val);
}
foreach ($args as $arg => $val) {
$elem->setAttribute($arg, $val);
}
$body->appendChild($elem);
return $elem;
}
private function processBundle($item, $dom, $body)
{
$page = getSimpleHTMLDOMCached($item['uri']);
$json_text = $page->find('#webpack-bundle-page-data', 0)->innertext;
$json = json_decode(html_entity_decode($json_text), true)['bundleData'];
$tiers = $json['tier_display_data'];
ksort($tiers, SORT_NATURAL);
# `initial` element gets sorted to the end as bt# (bundle tiers) precede it alphabetically
array_unshift($tiers, array_pop($tiers));
$seen = [];
$toc = $this->createChild($dom, $body, 'ul');
foreach ($tiers as $tiername => $tier) {
$this->createChild($dom, $body, 'h2', $tier['header'], ['id' => $tiername]);
$li = $this->createChild($dom, $toc, 'li');
$this->createChild($dom, $li, 'a', $tier['header'], ['href' => "#$tiername"]);
$toc_tier = $this->createChild($dom, $toc, 'ul');
foreach ($tier['tier_item_machine_names'] as $name) {
if (in_array($name, $seen)) {
continue;
}
array_push($seen, $name);
$element = $json['tier_item_data'][$name];
$head = $this->createChild($dom, $body, 'h3', null, ['id' => $name]);
$head_link = $this->createChild($dom, $head, 'a', $element['human_name'], ['id' => $name]);
$li = $this->createChild($dom, $toc_tier, 'li');
$this->createChild($dom, $li, 'a', $element['human_name'], ['href' => "#$name"]);
$this->createChild($dom, $body, 'img', null, ['src' => $element['resolved_paths']['featured_image']]);
$this->createChild($dom, $body, 'img', null, ['src' => $element['resolved_paths']['preview_image']]);
$this->createChild($dom, $body, 'br');
if ($element['description_text']) {
$body->appendChild(str_get_html($element['description_text'])->root);
}
if ($element['youtube_link']) {
$head_link->href = 'https://youtu.be/' . $element['youtube_link'];
}
if ($element['book_preview']) {
$head_link->href = $element['book_preview']['preview_file_link'];
}
}
}
return $item;
}
public function getName()
{
$name = parent::getName();

293
bridges/I4wifiBridge.php Normal file
View File

@@ -0,0 +1,293 @@
<?php
/**
*
* The website i4wifi.cz is a wholesale distributor specializing in wireless, networking, and photovoltaic equipment, offering products from brands like MikroTik, Ubiquiti, and Hikvision. It provides a wide range of network solutions, technical support, and training services for businesses and professional installers in the Czech Republic and beyond.
*/
class I4wifiBridge extends BridgeAbstract
{
const NAME = 'i4wifi Bridge';
const URI = 'https://www.i4wifi.cz';
const DESCRIPTION = 'Product news not only from the wireless, network and security technology sector from i4wifi.cz - Czech Republic';
const MAINTAINER = 'pprenghyorg';
// Only Articles are supported
const PARAMETERS = [
'Product news' => [
],
];
/**
* Fetches and processes data based on the selected context.
*
* This function retrieves the HTML content for the specified context's URI,
* resolves relative links within the content, and then delegates the data
* extraction to the appropriate method (currently only `collectNews`).
*/
public function collectData()
{
$html = getSimpleHTMLDOMCached($this->getURI(), 86400);
defaultLinkTo($html, static::URI);
// Router
switch ($this->queriedContext) {
case 'Product news':
$this->collectNews($html);
break;
}
}
/**
* Returns the icon for the bridge.
*
* @return string The icon URL.
*/
public function getURI()
{
$uri = static::URI;
// URI Router
switch ($this->queriedContext) {
case 'Product news':
$uri .= '/';
break;
}
return $uri;
}
/**
* Returns the name for the bridge.
*
* @return string The Name.
*/
public function getName()
{
$name = static::NAME;
$name .= ($this->queriedContext) ? ' - ' . $this->queriedContext : '';
switch ($this->queriedContext) {
case 'Product news':
break;
}
return $name;
}
/**
* Parse most used date formats
*
* Basically strtotime doesn't convert dates correctly due to formats
* being hard to interpret. So we use the DateTime object, manually
* fixing dates and times (set to 00:00:00.000).
*
* We don't know the timezone, so just assume +00:00 (or whatever
* DateTime chooses)
*/
private function fixDate($date)
{
$df = $this->parseDateTimeFromString($date);
return date_format($df, 'U');
}
/**
* Extracts the images from the article.
*
* @param object $article The article object.
* @return array An array of image URLs.
*/
private function extractImages($article)
{
// Notice: We can have zero or more images (though it should mostly be 1)
$elements = $article->find('img');
$images = [];
foreach ($elements as $img) {
$images[] = $img->src;
}
return $images;
}
#region Articles
/**
* Collects uri, timestamp, title, content and images in the news articles from the HTML and transforms to rss.
*
* @param object $html The HTML object.
* @return void
*/
private function collectNews($html)
{
$articles = $html->find('.timeline-item.timeline-item-right')
or throwServerException('No articles found! Layout might have changed!');
foreach ($articles as $article) {
$item = [];
// get uri of product
$item['uri'] = $this->extractNewsUri($article);
// Add content
$item['content'] = $this->extractNewsDescription($article);
// Add images
$item['title'] = $this->extractNewsTitle($article);
// Add images
$item['enclosures'] = $this->extractImages($article);
// Add timestamp
$item['timestamp'] = $this->extractNewsDate($article);
// collect sources into rss article
$this->items[] = $item;
}
}
/**
* Extracts the URI of the news article.
*
* @param object $article The article object.
* @return string The URI of the news article.
*/
private function extractNewsUri($article)
{
// Return URI of the article
$element = $article->find('a', 0)
or throwServerException('Anchor not found!');
return $element->href;
}
/**
* Extracts the date of the news article.
*
* @param object $article The article object.
* @return string The date of the news article.
*/
private function extractNewsDate($article)
{
// Check if date is set
$element = $article->find('.timeline-item-info', 0)
or throwServerException('Date not found!');
// Format date
return $this->fixDate($element->plaintext);
}
/**
* Extracts the description of the news article.
*
* @param object $article The article object.
* @return string The description of the news article.
*/
private function extractNewsDescription($article)
{
// Extract description
$element = $article->find('p', 0)
or throwServerException('Description not found!');
return $element->innertext;
}
/**
* Extracts the title of the news article.
*
* @param object $article The article object.
* @return string The title of the news article.
*/
private function extractNewsTitle($article)
{
// Extract title
$element = $article->find('img', 0)
or throwServerException('Title not found!');
return $element->alt;
}
/**
* It attempts to recognize the date/time format in a string and create a DateTime object.
*
* It goes through the list of defined formats and tries to apply them to the input string.
* Returns the first successfully parsed DateTime object that matches the entire string.
*
* @param string $dateString A string potentially containing a date and/or time.
* @return DateTime|null A DateTime object if successfully recognized and parsed, otherwise null.
*/
private function parseDateTimeFromString(string $dateString): ?DateTime
{
// List of common formats - YOU CAN AND SHOULD EXPAND IT according to expected inputs!
// Order may matter if the formats are ambiguous.
// It is recommended to give more specific formats (with time, full year) before more general ones.
$possibleFormats = [
// Czech formats (day.month.year)
'd.m.Y H:i:s', // 10.04.2025 10:57:47
'j.n.Y H:i:s', // 10.4.2025 10:57:47
'd. m. Y H:i:s', // 10. 04. 2025 10:57:47
'j. n. Y H:i:s', // 10. 4. 2025 10:57:47
'd.m.Y H:i', // 10.04.2025 10:57
'j.n.Y H:i', // 10.4.2025 10:57
'd. m. Y H:i', // 10. 04. 2025 10:57
'j. n. Y H:i', // 10. 4. 2025 10:57
'd.m.Y', // 10.04.2025
'j.n.Y', // 10.4.2025
'd. m. Y', // 10. 04. 2025
'j. n. Y', // 10. 4. 2025
// ISO 8601 and international formats (year-month-day)
'Y-m-d H:i:s', // 2025-04-10 10:57:47
'Y-m-d H:i', // 2025-04-10 10:57
'Y-m-d', // 2025-04-10
'YmdHis', // 20250410105747
'Ymd', // 20250410
// American formats (month/day/year) - beware of ambiguity!
'm/d/Y H:i:s', // 04/10/2025 10:57:47
'n/j/Y H:i:s', // 4/10/2025 10:57:47
'm/d/Y H:i', // 04/10/2025 10:57
'n/j/Y H:i', // 4/10/2025 10:57
'm/d/Y', // 04/10/2025
'n/j/Y', // 4/10/2025
// Standard formats (including time zone)
DateTime::ATOM, // example. 2025-04-10T10:57:47+02:00
DateTime::RFC3339, // example. 2025-04-10T10:57:47+02:00
DateTime::RFC3339_EXTENDED, // example. 2025-04-10T10:57:47.123+02:00
DateTime::RFC2822, // example. Thu, 10 Apr 2025 10:57:47 +0200
DateTime::ISO8601, // example. 2025-04-10T105747+0200
'Y-m-d\TH:i:sP', // ISO 8601 s 'T' oddělovačem
'Y-m-d\TH:i:s.uP', // ISO 8601 s mikrosekundami
// You can add more formats as needed...
// e.g. 'd-M-Y' (10-Apr-2025) - requires English locale
// e.g. 'j. F Y' (10. abren 2025) - requires Czech locale
];
// Set locale for parsing month/day names (if using F, M, l, D)
// E.g. setlocale(LC_TIME, 'cs_CZ.UTF-8'); or 'en_US.UTF-8');
foreach ($possibleFormats as $format) {
// We will try to create a DateTime object from the given format
$dateTime = DateTime::createFromFormat($format, $dateString);
// We check that the parsing was successful AND ALSO
// that there were no errors or warnings during the parsing.
// This is important to ensure that the format matches the ENTIRE string.
if ($dateTime !== false) {
$errors = DateTime::getLastErrors();
if (!($errors)) {
// Success! We found a valid format for the entire string.
return $dateTime;
}
}
}
// If no format matches or parsing failed
return null;
}
#endregion
}

View File

@@ -44,7 +44,7 @@ class IPBBridge extends FeedExpander
switch (parse_url($this->getInput('uri'), PHP_URL_PATH)) {
case null:
case '/index.php':
returnClientError('Provided URI is invalid!');
throwClientException('Provided URI is invalid!');
break;
default:
break;
@@ -75,7 +75,7 @@ class IPBBridge extends FeedExpander
$this->collectForum($html);
break;
default:
returnClientError('Unknown type!');
throwClientException('Unknown type!');
break;
}
}
@@ -106,7 +106,7 @@ class IPBBridge extends FeedExpander
$this->collectForumTable($html);
break;
default:
returnClientError('Unknown forum format!');
throwClientException('Unknown forum format!');
break;
}
}
@@ -159,7 +159,7 @@ class IPBBridge extends FeedExpander
$this->collectTopicHistory($html, $limit, 'collectTopicDiv');
break;
default:
returnClientError('Unknown topic format!');
throwClientException('Unknown topic format!');
break;
}
}
@@ -168,7 +168,7 @@ class IPBBridge extends FeedExpander
{
// Make sure the callback is valid!
if (!method_exists($this, $callback)) {
returnServerError('Unknown function (\'' . $callback . '\')!');
throwServerException('Unknown function (\'' . $callback . '\')!');
}
$next = null; // Holds the URI of the next page

View File

@@ -35,6 +35,16 @@ class IdealoBridge extends BridgeAbstract
]
];
private $headers = [
'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:139.0) Gecko/20100101 Firefox/139.0',
'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8',
'Accept-Language: fr-FR,fr;q=0.8,en-US;q=0.5,en;q=0.3'
];
private $options = [
CURLOPT_TRANSFER_ENCODING => 1,
CURLOPT_ACCEPT_ENCODING => 'gzip, deflate, br'
];
public function getIcon()
{
return 'https://cdn.idealo.com/storage/ids-assets/ico/favicon.ico';
@@ -53,10 +63,7 @@ class IdealoBridge extends BridgeAbstract
// The cache does not contain the title of the bridge, we must get it and save it in the cache
if ($product === null) {
$header = [
'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2.1 Safari/605.1.15'
];
$html = getSimpleHTMLDOM($link, $header);
$html = getSimpleHTMLDOM($link, $this->headers, $this->options);
$product = $html->find('.oopStage-title', 0)->find('span', 0)->plaintext;
$this->saveCacheValue($keyTITLE, $product);
}
@@ -123,13 +130,8 @@ class IdealoBridge extends BridgeAbstract
}
public function collectData()
{
// Needs header with user-agent to function properly.
$header = [
'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2.1 Safari/605.1.15'
];
$link = $this->getInput('Link');
$html = getSimpleHTMLDOM($link, $header);
$html = getSimpleHTMLDOM($link, $this->headers, $this->options);
// Get Productname
$titleobj = $html->find('.oopStage-title', 0);

View File

@@ -59,7 +59,7 @@ class ImgsedBridge extends BridgeAbstract
$this->collectTaggeds();
}
} catch (HttpException $e) {
throw new \Exception(sprintf('Unable to find user `%s`', $username));
throwClientException(sprintf('Unable to find user `%s`', $username));
}
}
@@ -258,7 +258,7 @@ HTML,
// If no content type is selected, this bridge does nothing, so we return an error
if (count($types) == 0) {
returnClientError('You must select at least one of the content type : Post, Stories or Tags !');
throwClientException('You must select at least one of the content type : Post, Stories or Tags !');
}
$typesText = $types[0] ?? '';

View File

@@ -39,7 +39,7 @@ class InstituteForTheStudyOfWarBridge extends BridgeAbstract
list($date_string, $user) = explode('-', $date_span->innertext);
$date = DateTime::createFromFormat('F d, Y', trim($date_string));
$html = getSimpleHTMLDOMCached(self::URI . $uri);
$html = getSimpleHTMLDOMCached(self::URI . $uri, 60 * 60 * 24 * 7);
$content = $html->find('[property=content:encoded]', 0)->innertext;
$enclosures = [];

View File

@@ -443,7 +443,7 @@ class ItakuBridge extends BridgeAbstract
return $data['owner'];
}
private function getPost($id, array $metadata = null)
private function getPost($id, ?array $metadata = null)
{
if (isset($metadata) && count($metadata['gallery_images']) < $metadata['num_images']) {
$metadata = null; //force re-fetch of metadata
@@ -515,7 +515,7 @@ class ItakuBridge extends BridgeAbstract
];
}
private function getCommission($id, array $metadata = null)
private function getCommission($id, ?array $metadata = null)
{
$url = self::URI . '/api/commissions/' . $id . '/?format=json';
$uri = self::URI . '/commissions/' . $id;
@@ -689,7 +689,7 @@ class ItakuBridge extends BridgeAbstract
if (is_array($item) || is_object($item)) {
$this->items[] = $item;
} else {
returnServerError("Incorrectly parsed item. Check the code!\nType: " . gettype($item) . "\nprint_r(item:)\n" . var_dump($item));
throwServerException("Incorrectly parsed item. Check the code!\nType: " . gettype($item) . "\nprint_r(item:)\n" . var_dump($item));
}
}
}

View File

@@ -21,7 +21,7 @@ class ItchioBridge extends BridgeAbstract
$html = getSimpleHTMLDOM($url);
// if the page is password protected, abort
if ($html->find('.game_password_page', 0) !== null) {
returnClientError('The requested page is password protected.');
throwClientException('The requested page is password protected.');
}
$title = $html->find('.game_title', 0)->innertext;

View File

@@ -110,7 +110,7 @@ class IvooxBridge extends BridgeAbstract
$this->request = str_replace(' ', '-', $this->getInput('s'));
$url_feed = self::URI . urlencode($this->request) . '_sb_f_1.html?o=uploaddate';
} else {
returnClientError('Not valid mode at IvooxBridge');
throwClientException('Not valid mode at IvooxBridge');
}
$dom = getSimpleHTMLDOM($url_feed);

View File

@@ -167,7 +167,7 @@ class JustETFBridge extends BridgeAbstract
private function collectNews($html)
{
$articles = $html->find('div.newsTopArticle')
or returnServerError('No articles found! Layout might have changed!');
or throwServerException('No articles found! Layout might have changed!');
foreach ($articles as $article) {
$item = [];
@@ -184,7 +184,7 @@ class JustETFBridge extends BridgeAbstract
$html = getSimpleHTMLDOMCached($uri);
$fullArticle = $html->find('div.article', 0)
or returnServerError('No content found! Layout might have changed!');
or throwServerException('No content found! Layout might have changed!');
defaultLinkTo($fullArticle, static::URI);
@@ -203,7 +203,7 @@ class JustETFBridge extends BridgeAbstract
private function extractNewsUri($article)
{
$element = $article->find('a', 0)
or returnServerError('Anchor not found!');
or throwServerException('Anchor not found!');
return $element->href;
}
@@ -211,7 +211,7 @@ class JustETFBridge extends BridgeAbstract
private function extractNewsDate($article)
{
$element = $article->find('div.subheadline', 0)
or returnServerError('Date not found!');
or throwServerException('Date not found!');
$date = trim(explode('|', $element->plaintext)[0]);
@@ -221,7 +221,7 @@ class JustETFBridge extends BridgeAbstract
private function extractNewsDescription($article)
{
$element = $article->find('span.newsText', 0)
or returnServerError('Description not found!');
or throwServerException('Description not found!');
$element->find('a', 0)->onclick = '';
@@ -231,7 +231,7 @@ class JustETFBridge extends BridgeAbstract
private function extractNewsTitle($article)
{
$element = $article->find('h3', 0)
or returnServerError('Title not found!');
or throwServerException('Title not found!');
return $element->plaintext;
}
@@ -239,7 +239,7 @@ class JustETFBridge extends BridgeAbstract
private function extractFullArticleContent($article)
{
$element = $article->find('div.article_body', 0)
or returnServerError('Article body not found!');
or throwServerException('Article body not found!');
// Remove teaser image
$element->find('img.teaser-img', 0)->outertext = '';
@@ -266,7 +266,7 @@ class JustETFBridge extends BridgeAbstract
private function extractFullArticleAuthor($article)
{
$element = $article->find('span[itemprop=name]', 0)
or returnServerError('Author not found!');
or throwServerException('Author not found!');
return $element->plaintext;
}
@@ -291,7 +291,7 @@ class JustETFBridge extends BridgeAbstract
private function extractProfileDate($html)
{
$element = $html->find('div.infobox div.vallabel', 0)
or returnServerError('Date not found!');
or throwServerException('Date not found!');
$date = trim(explode("\r\n", $element->plaintext)[1]);
@@ -301,7 +301,7 @@ class JustETFBridge extends BridgeAbstract
private function extractProfileTitle($html)
{
$element = $html->find('span.h1', 0)
or returnServerError('Title not found!');
or throwServerException('Title not found!');
return $element->plaintext;
}
@@ -314,12 +314,12 @@ class JustETFBridge extends BridgeAbstract
// - Quote
$strategy = $html->find('div.tab-container div.col-sm-6 p', 0)
or returnServerError('Investment Strategy not found!');
or throwServerException('Investment Strategy not found!');
// Description requires a bit of cleanup due to lack of propper identification
$description = $html->find('div.headline', 5)
or returnServerError('Description container not found!');
or throwServerException('Description container not found!');
$description = $description->parent();
@@ -328,7 +328,7 @@ class JustETFBridge extends BridgeAbstract
}
$quote = $html->find('div.infobox div.val', 0)
or returnServerError('Quote not found!');
or throwServerException('Quote not found!');
$quote_html = '<strong>Quote</strong><br><p>' . $quote . '</p>';
$strategy_html = '';
@@ -350,7 +350,7 @@ class JustETFBridge extends BridgeAbstract
// Use ISIN + WKN as author
// Notice: "identfier" is not a typo [sic]!
$element = $html->find('span.identfier', 0)
or returnServerError('Author not found!');
or throwServerException('Author not found!');
return $element->plaintext;
}

View File

@@ -24,6 +24,11 @@ class KemonoBridge extends BridgeAbstract
'name' => 'User ID/Name',
'exampleValue' => '9069743', # Thomas Joy
'required' => true,
],
'q' => [
'name' => 'Search query',
'exampleValue' => 'classic',
'required' => false,
]
]];
@@ -33,13 +38,17 @@ class KemonoBridge extends BridgeAbstract
{
$api = parent::getURI() . 'api/v1/';
$url = $api . $this->getInput('service') . '/user/' . $this->getInput('user');
$api_response = getContents($url . '/profile');
$profile = Json::decode($api_response);
$this->title = ucfirst($profile['name']);
if ($this->getInput('q')) {
$url .= '?q=' . urlencode($this->getInput('q'));
}
$api_response = getContents($url);
$json = Json::decode($api_response);
$url .= '/profile';
$api_response = getContents($url);
$profile = Json::decode($api_response);
$this->title = ucfirst($profile['name']);
foreach ($json as $element) {
$item = [];

View File

@@ -420,7 +420,7 @@ class LaCentraleBridge extends BridgeAbstract
!empty($this->getInput('distance'))
&& is_null($this->getInput('location'))
) {
returnClientError('You need a place ("CP ou département") to search arround.');
throwClientException('You need a place ("CP ou département") to search arround.');
}
$params = [

View File

@@ -339,14 +339,14 @@ class LeBonCoinBridge extends BridgeAbstract
&& !is_null($range_max)
&& $range_min > $range_max
) {
returnClientError('Min-' . $field . ' must be lower than max-' . $field . '.');
throwClientException('Min-' . $field . ' must be lower than max-' . $field . '.');
}
if (
!is_null($range_min)
&& is_null($range_max)
) {
returnClientError('Max-' . $field . ' is needed when min-' . $field . ' is setted (range).');
throwClientException('Max-' . $field . ' is needed when min-' . $field . ' is setted (range).');
}
return [

View File

@@ -16,7 +16,7 @@ class LinuxBlogBridge extends BridgeAbstract
$articles = $dom->find('ul.display-posts-listing li.listing-item');
if (!$articles) {
returnServerError('Failed to retrieve articles');
throwServerException('Failed to retrieve articles');
}
foreach ($articles as $article) {

View File

@@ -29,7 +29,7 @@ class MallTvBridge extends BridgeAbstract
$scriptLdJson = $html->find('script[type="application/ld+json"]', 0)->innertext;
if (!preg_match('/[\'"]uploadDate[\'"]\s*:\s*[\'"](\d{4}-\d{2}-\d{2})[\'"]/', $scriptLdJson, $match)) {
returnServerError('Could not get date from MALL.TV detail page');
throwServerException('Could not get date from MALL.TV detail page');
}
return strtotime($match[1]);
@@ -40,7 +40,7 @@ class MallTvBridge extends BridgeAbstract
$url = $this->getInput('url');
if (!preg_match('/^https:\/\/www\.mall\.tv\/[a-z0-9-]+(\/[a-z0-9-]+)?\/?$/', $url)) {
returnServerError('Invalid url');
throwServerException('Invalid url');
}
$html = getSimpleHTMLDOM($url);

View File

@@ -108,7 +108,7 @@ class MangaDexBridge extends BridgeAbstract
switch ($this->queriedContext) {
case 'Title Chapters':
preg_match(self::TITLE_REGEX, $this->getInput('url'), $matches)
or returnClientError('Invalid URL Parameter');
or throwClientException('Invalid URL Parameter');
$this->feedURI = self::URI . 'title/' . $matches['uuid'];
$params['order[readableAt]'] = 'desc';
if (!$this->getInput('external')) {
@@ -129,7 +129,7 @@ class MangaDexBridge extends BridgeAbstract
$uri = self::API_ROOT . 'chapter';
break;
default:
returnServerError('Unimplemented Context (getAPI)');
throwServerException('Unimplemented Context (getAPI)');
}
// Remove null keys
@@ -180,7 +180,7 @@ class MangaDexBridge extends BridgeAbstract
if ($content['result'] == 'ok') {
$content = $content['data'];
} else {
returnServerError('Could not retrieve API results');
throwServerException('Could not retrieve API results');
}
switch ($this->queriedContext) {
@@ -191,7 +191,7 @@ class MangaDexBridge extends BridgeAbstract
$this->getChapters($content);
break;
default:
returnServerError('Unimplemented Context (collectData)');
throwServerException('Unimplemented Context (collectData)');
}
}
@@ -257,7 +257,7 @@ class MangaDexBridge extends BridgeAbstract
$header = [ 'Content-Type: application/json' ];
$pages = json_decode(getContents($api_uri, $header), true);
if ($pages['result'] != 'ok') {
returnServerError('Could not retrieve API results');
throwServerException('Could not retrieve API results');
}
if ($this->getInput('images') == 'saver') {

View File

@@ -21,7 +21,7 @@ class MinecraftBridge extends BridgeAbstract
$articles = json_decode($json);
if ($articles === null) {
returnServerError('Failed to decode JSON content.');
throwServerException('Failed to decode JSON content.');
}
foreach ($articles->article_grid as $article) {

View File

@@ -22,7 +22,7 @@ class ModelKarteiBridge extends BridgeAbstract
{
$model_id = preg_replace('/[^0-9]/', '', $this->getInput('model_id'));
if (empty($model_id)) {
returnServerError('Invalid model ID');
throwServerException('Invalid model ID');
}
$html = getSimpleHTMLDOM(self::URI . 'sedcards/model/' . $model_id . '/');
@@ -34,7 +34,7 @@ class ModelKarteiBridge extends BridgeAbstract
$itemlist = $html->find('#photoList .photoPreview');
if (!$itemlist) {
returnServerError('No gallery');
throwServerException('No gallery');
}
foreach ($itemlist as $idx => $element) {

126
bridges/ModrinthBridge.php Normal file
View File

@@ -0,0 +1,126 @@
<?php
declare(strict_types=1);
// Uses the modrinth API documented here: https://docs.modrinth.com/api/
class ModrinthBridge extends BridgeAbstract
{
const NAME = 'Modrinth';
const URI = 'https://modrinth.com/';
const DESCRIPTION = 'For new versions of mods, resource packs, etc.';
const MAINTAINER = 'xnand';
const PARAMETERS = [[
'name' => [
'name' => 'Name',
'required' => true,
'title' => 'The project name as seen in the URL bar',
'exampleValue' => 'sodium'
],
'category' => [
'name' => 'Category',
'type' => 'list',
'values' => [
'Mod' => 'mod',
'Resource Pack' => 'resourcepack',
'Data Pack' => 'datapack',
'Shader' => 'shader',
'Modpack' => 'modpack',
'Plugin' => 'plugin'
],
'defaultValue' => 'mod'
],
'loaders' => [
'name' => 'Loaders',
'title' => 'List of mod loaders, separated by commas',
'exampleValue' => 'neoforge, fabric'
],
'game_versions' => [
'name' => 'Game versions',
'title' => 'List of game versions, separated by commas',
'exampleValue' => '1.19.1, 1.19.2'
],
'featured' => [
'name' => 'Featured',
'type' => 'list',
'values' => [
'Unset' => '',
'True' => 'true',
'False' => 'false'
],
'title' => "Whether to filter for featured or non-featured\nUnset means no filter",
'defaultValue', ''
]
]];
public function getURI()
{
$name = $this->getInput('name');
$category = $this->getInput('category');
$uri = self::URI . $category . '/' . $name . '/versions';
if (empty($name)) {
$uri = parent::getURI();
}
return $uri;
}
public function getName()
{
$name = $this->getInput('name');
if (empty($name)) {
$name = parent::getName();
}
return $name;
}
public function collectData()
{
$apiUrl = 'https://api.modrinth.com/v2/project';
$projectName = $this->getInput('name');
$url = "{$apiUrl}/${projectName}/version";
$queryTable = [
'loaders' => $this->parseInputList($this->getInput('loaders')),
'game_versions' => $this->parseInputList($this->getInput('game_versions')),
'featured' => ($this->getInput('featured')) ? : null
];
$query = http_build_query($queryTable);
if ($query) {
$url .= '?' . $query;
}
// They expect a descriptive user agent and may block connections without one
// Change as appropriate
// https://docs.modrinth.com/api/#user-agents
$header = [ 'User-Agent: rss-bridge plugin https://github.com/RSS-Bridge/rss-bridge' ];
$data = json_decode(getContents($url, $header));
foreach ($data as $entry) {
$item = [];
$item['uri'] = self::URI . $this->getInput('category') . '/' . $this->getInput('name') . '/version/' . $entry->version_number;
$item['title'] = $entry->name;
$item['timestamp'] = $entry->date_published;
// Not setting the author as this would take a second request to match the author's user ID
$item['author'] = 'Modrinth';
$item['content'] = markdownToHtml($entry->changelog);
$item['categories'] = array_merge($entry->loaders, $entry->game_versions);
$item['uid'] = $entry->id;
$this->items[] = $item;
}
}
// Converts lists like `foo, bar, baz` to `["foo", "bar", "baz"]`
protected function parseInputList($input): ?string
{
if (empty($input)) {
return null;
}
$items = array_filter(array_map('trim', explode(',', $input)));
return $items ? json_encode($items) : null; // return nothing if string is empty
}
}

View File

@@ -166,7 +166,7 @@ class MoinMoinBridge extends BridgeAbstract
private function splitSections($html)
{
$content = $html->find('div#page', 0)->innertext
or returnServerError('Unable to find <div id="page"/>!');
or throwServerException('Unable to find <div id="page"/>!');
$sections = [];

View File

@@ -0,0 +1,291 @@
<?php
/**
*
* NaseStrecha.cz is a specialized Czech news and advice portal focusing on roofs, construction, and home improvement, offering reliable expert guidance on roofing materials, insulation, and energy-saving techniques nasestrecha.cz . It is run by the team behind the Strechy-Solar-Remeslo trade fair and includes up-to-date news, practical tips, and industry events..
*
*/
class NasestrechaBridge extends BridgeAbstract
{
const NAME = 'Nasestrecha Bridge';
const URI = 'https://www.nasestrecha.cz/';
const DESCRIPTION = 'Articles from Nasestrecha.cz news site - Czech Republic / Spolehlivé informace pro Vaší střechu i stavbu';
const MAINTAINER = 'pprenghyorg';
// Only Articles are supported
const PARAMETERS = [
'Articles, news and reviews from from construction and housing' => [
],
];
/**
* Fetches and processes data based on the selected context.
*
* This function retrieves the HTML content for the specified context's URI,
* resolves relative links within the content, and then delegates the data
* extraction to the appropriate method (currently only `collectNews`).
*/
public function collectData()
{
$html = getSimpleHTMLDOM($this->getURI());
defaultLinkTo($html, static::URI);
// Router
switch ($this->queriedContext) {
case 'Articles, news and reviews from from construction and housing':
$this->collectNews($html);
break;
}
}
/**
* Returns the icon for the bridge.
*
* @return string The icon URL.
*/
public function getURI()
{
$uri = static::URI;
// URI Router
switch ($this->queriedContext) {
case 'Articles, news and reviews from from construction and housing':
$uri .= 'clanky/';
break;
}
return $uri;
}
/**
* Returns the name for the bridge.
*
* @return string The Name.
*/
public function getName()
{
$name = static::NAME;
$name .= ($this->queriedContext) ? ' - ' . $this->queriedContext : '';
switch ($this->queriedContext) {
case 'Articles, news and reviews from from construction and housing':
break;
}
return $name;
}
/**
* Parse most used date formats
*
* Basically strtotime doesn't convert dates correctly due to formats
* being hard to interpret. So we use the DateTime object, manually
* fixing dates and times (set to 00:00:00.000).
*
* We don't know the timezone, so just assume +00:00 (or whatever
* DateTime chooses)
*/
private function fixDate($date)
{
$df = $this->parseDateTimeFromString($date);
return date_format($df, 'U');
}
/**
* Extracts the images from the article.
*
* @param object $article The article object.
* @return array An array of image URLs.
*/
private function extractImages($article)
{
// Notice: We can have zero or more images (though it should mostly be 1)
$elements = $article->find('img');
$images = [];
foreach ($elements as $img) {
$images[] = $img->src;
}
return $images;
}
#region Articles
/**
* Collects uri, timestamp, title, content and images in the news articles from the HTML and transforms to rss.
*
* @param object $html The HTML object.
* @return void
*/
private function collectNews($html)
{
// Check if page contains articles
$articles = $html->find('.post')
or throwServerException('No articles found! Layout might have changed!');
foreach ($articles as $article) {
$item = [];
$item['uri'] = $this->extractNewsUri($article);
$item['timestamp'] = $this->extractNewsDate($article);
$item['title'] = $this->extractNewsTitle($article);
$item['content'] = $this->extractNewsDescription($article);
$item['enclosures'] = $this->extractImages($article);
// collect sources into rss article
$this->items[] = $item;
}
}
/**
* Extracts the URI of the news article.
*
* @param object $article The article object.
* @return string The URI of the news article.
*/
private function extractNewsUri($article)
{
// Return URI of the article
$element = $article->find('.thumbnail', 0)
or throwServerException('Anchor not found!');
return $element->href;
}
/**
* Extracts the date of the news article.
*
* @param object $article The article object.
* @return string The date of the news article.
*/
private function extractNewsDate($article)
{
// Check if date is set
$element = $article->find('div.post__info', 0)->find('span', 0)
or throwServerException('Date not found!');
$date = trim(explode('|', $element->plaintext)[0]);
// Format date
return $this->fixDate($date);
}
/**
* Extracts the description of the news article.
*
* @param object $article The article object.
* @return string The description of the news article.
*/
private function extractNewsDescription($article)
{
// Extract description
$element = $article->find('p.post__text', 0)
or throwServerException('Description not found!');
return $element->innertext;
}
/**
* Extracts the title of the news article.
*
* @param object $article The article object.
* @return string The title of the news article.
*/
private function extractNewsTitle($article)
{
// Extract title
$element = $article->find('a.post__title', 0)
or throwServerException('Title not found!');
return $element->plaintext;
}
/**
* It attempts to recognize the date/time format in a string and create a DateTime object.
*
* It goes through the list of defined formats and tries to apply them to the input string.
* Returns the first successfully parsed DateTime object that matches the entire string.
*
* @param string $dateString A string potentially containing a date and/or time.
* @return DateTime|null A DateTime object if successfully recognized and parsed, otherwise null.
*/
private function parseDateTimeFromString(string $dateString): ?DateTime
{
// List of common formats - YOU CAN AND SHOULD EXPAND IT according to expected inputs!
// Order may matter if the formats are ambiguous.
// It is recommended to give more specific formats (with time, full year) before more general ones.
$possibleFormats = [
// Czech formats (day.month.year)
'd.m.Y H:i:s', // 10.04.2025 10:57:47
'j.n.Y H:i:s', // 10.4.2025 10:57:47
'd. m. Y H:i:s', // 10. 04. 2025 10:57:47
'j. n. Y H:i:s', // 10. 4. 2025 10:57:47
'd.m.Y H:i', // 10.04.2025 10:57
'j.n.Y H:i', // 10.4.2025 10:57
'd. m. Y H:i', // 10. 04. 2025 10:57
'j. n. Y H:i', // 10. 4. 2025 10:57
'd.m.Y', // 10.04.2025
'j.n.Y', // 10.4.2025
'd. m. Y', // 10. 04. 2025
'j. n. Y', // 10. 4. 2025
// ISO 8601 and international formats (year-month-day)
'Y-m-d H:i:s', // 2025-04-10 10:57:47
'Y-m-d H:i', // 2025-04-10 10:57
'Y-m-d', // 2025-04-10
'YmdHis', // 20250410105747
'Ymd', // 20250410
// American formats (month/day/year) - beware of ambiguity!
'm/d/Y H:i:s', // 04/10/2025 10:57:47
'n/j/Y H:i:s', // 4/10/2025 10:57:47
'm/d/Y H:i', // 04/10/2025 10:57
'n/j/Y H:i', // 4/10/2025 10:57
'm/d/Y', // 04/10/2025
'n/j/Y', // 4/10/2025
// Standard formats (including time zone)
DateTime::ATOM, // example. 2025-04-10T10:57:47+02:00
DateTime::RFC3339, // example. 2025-04-10T10:57:47+02:00
DateTime::RFC3339_EXTENDED, // example. 2025-04-10T10:57:47.123+02:00
DateTime::RFC2822, // example. Thu, 10 Apr 2025 10:57:47 +0200
DateTime::ISO8601, // example. 2025-04-10T105747+0200
'Y-m-d\TH:i:sP', // ISO 8601 s 'T' oddělovačem
'Y-m-d\TH:i:s.uP', // ISO 8601 s mikrosekundami
// You can add more formats as needed...
// e.g. 'd-M-Y' (10-Apr-2025) - requires English locale
// e.g. 'j. F Y' (10. abren 2025) - requires Czech locale
];
// Set locale for parsing month/day names (if using F, M, l, D)
// E.g. setlocale(LC_TIME, 'cs_CZ.UTF-8'); or 'en_US.UTF-8');
foreach ($possibleFormats as $format) {
// We will try to create a DateTime object from the given format
$dateTime = DateTime::createFromFormat($format, $dateString);
// We check that the parsing was successful AND ALSO
// that there were no errors or warnings during the parsing.
// This is important to ensure that the format matches the ENTIRE string.
if ($dateTime !== false) {
$errors = DateTime::getLastErrors();
if (!($errors)) {
// Success! We found a valid format for the entire string.
return $dateTime;
}
}
}
// If no format matches or parsing failed
return null;
}
#endregion
}

View File

@@ -75,7 +75,7 @@ class NationalGeographicBridge extends BridgeAbstract
case self::TOPIC_LATEST_STORIES:
return $this->collectLatestStories();
default:
returnServerError('Unknown topic: "' . $this->topicName . '"');
throwServerException('Unknown topic: "' . $this->topicName . '"');
}
}

View File

@@ -117,7 +117,7 @@ class OtrkeyFinderBridge extends BridgeAbstract
// Do we need to check the running time?
if ($minTime != 0 || $maxTime != 0) {
if ($maxTime > 0 && $maxTime < $minTime) {
returnClientError('The minimum running time must be less than the maximum running time.');
throwClientException('The minimum running time must be less than the maximum running time.');
}
preg_match(self::FILENAME_REGEX, $file, $matches);

View File

@@ -25,7 +25,7 @@ class PatreonBridge extends BridgeAbstract
if (preg_match($regex, $html->save(), $matches) > 0) {
$campaign_id = $matches[1];
} else {
returnServerError('Could not find campaign ID');
throwServerException('Could not find campaign ID');
}
$query = [

View File

@@ -25,22 +25,40 @@ class PcGamerBridge extends BridgeAbstract
$articleHtml = getSimpleHTMLDOMCached($item['uri']);
// Relying on meta tags ought to be more reliable.
$item['title'] = $articleHtml->find('meta[name=parsely-title]', 0)->content;
$item['title'] = $articleHtml->find('meta[property=og:title]', 0)->content;
$item['content'] = html_entity_decode($articleHtml->find('meta[name=description]', 0)->content);
$item['author'] = $articleHtml->find('meta[name=parsely-author]', 0)->content;
$imageUrl = $articleHtml->find('meta[name=parsely-image-url]', 0);
// TODO: parsely-author is no longer available, but it is in the application/ld+json
$item['author'] = $articleHtml->find('a[rel=author]', 0)->innertext;
$imageUrl = $articleHtml->find('meta[property=og:image]', 0);
if ($imageUrl) {
$item['enclosures'][] = $imageUrl->content;
}
/* I don't know why every article has two extra tags, but because
one matches another common tag, "guide," it needs to be removed. */
$item['categories'] = array_diff(
explode(',', $articleHtml->find('meta[name=parsely-tags]', 0)->content),
['van_buying_guide_progressive', 'serversidehawk']
/*
Tags in mrf:tags are semicolon-delimited and each begins with a label and a ':'
Example:
"region:US;articleType:News;channel:Gaming software;"
Find the tag, replace ; with \n, remove the label prefixes, then explode by newline.
*/
$item['categories'] = array_unique(
explode(
PHP_EOL,
preg_replace(
'/^[^:]+:/m',
'',
preg_replace(
'/;/',
PHP_EOL,
$articleHtml->find('meta[property=mrf:tags]', 0)->content
)
)
)
);
$item['timestamp'] = strtotime($articleHtml->find('meta[name=pub_date]', 0)->content);
$this->items[] = $item;
}
}

View File

@@ -62,7 +62,8 @@ class PepperBridgeAbstract extends BridgeAbstract
foreach ($list as $deal) {
// Get the JSON Data stored as vue
$jsonDealData = $this->getDealJsonData($deal);
$dealMeta = Json::decode($deal->find('div[class=js-vue2]', 1)->getAttribute('data-vue2'));
// DEPRECATED : website does not show this info in the deal list anymore
// $dealMeta = Json::decode($deal->find('div[class=js-vue3]', 1)->getAttribute('data-vue3'));
$item = [];
$item['uri'] = $this->getDealURI($jsonDealData);
@@ -77,7 +78,10 @@ class PepperBridgeAbstract extends BridgeAbstract
. $this->getHTMLTitle($jsonDealData)
. $this->getPrice($jsonDealData)
. $this->getDiscount($jsonDealData)
. $this->getShipsFrom($dealMeta)
/*
* DEPRECATED : the list does not show this info anymore
* . $this->getShipsFrom($dealMeta)
*/
. $this->getShippingCost($jsonDealData)
. $this->getSource($jsonDealData)
. $this->getDealLocation($jsonDealData)
@@ -105,7 +109,7 @@ class PepperBridgeAbstract extends BridgeAbstract
// Show an error message if we can't find the thread ID in the URL sent by the user
if ($threadSearch !== 1) {
returnClientError($this->i8n('thread-error'));
throwClientException($this->i8n('thread-error'));
}
$threadID = $matches[1];
@@ -354,7 +358,7 @@ HEREDOC;
*/
private function getDealJsonData($deal)
{
$data = Json::decode($deal->find('div[class=js-vue2]', 0)->getAttribute('data-vue2'));
$data = Json::decode($deal->find('div[class=js-vue3]', 0)->getAttribute('data-vue3'));
return $data;
}
@@ -419,7 +423,7 @@ HEREDOC;
private function getImage($deal)
{
// Get thread Image JSON content
$content = Json::decode($deal->find('div[class=js-vue2]', 0)->getAttribute('data-vue2'));
$content = Json::decode($deal->find('div[class=js-vue3]', 0)->getAttribute('data-vue3'));
//return '<img src="' . $content['props']['threadImageUrl'] . '"/>';
return '<img src="' . $this->i8n('image-host') . $content['props']['thread']['mainImage']['path'] . '/'
. $content['props']['thread']['mainImage']['name'] . '/re/202x202/qt/70/'
@@ -429,6 +433,7 @@ HEREDOC;
/**
* Get the originating country from a Deal if it exists
* @return string String of the deal originating country
* DEPRECATED : the deal on the result list does not contain this info anymore
*/
private function getShipsFrom($dealMeta)
{

View File

@@ -131,7 +131,7 @@ class PixivBridge extends BridgeAbstract
. '/profile/top';
break;
default:
returnClientError('Invalid Context');
throwClientException('Invalid Context');
}
return $uri;
}
@@ -279,7 +279,7 @@ class PixivBridge extends BridgeAbstract
if (
!(strlen($proxy) > 0 && preg_match('/https?:\/\/.*/', $proxy))
) {
returnServerError('Invalid proxy_url value set. The proxy must include the HTTP/S at the beginning of the url.');
throwServerException('Invalid proxy_url value set. The proxy must include the HTTP/S at the beginning of the url.');
}
}

View File

@@ -116,12 +116,12 @@ class RedditBridge extends BridgeAbstract
{
$forbiddenKey = 'reddit_forbidden';
if ($this->cache->get($forbiddenKey)) {
throw new RateLimitException();
throwRateLimitException();
}
$rateLimitKey = 'reddit_rate_limit';
if ($this->cache->get($rateLimitKey)) {
throw new RateLimitException();
throwRateLimitException();
}
try {
@@ -131,10 +131,10 @@ class RedditBridge extends BridgeAbstract
// 403 Forbidden
// This can possibly mean that reddit has permanently blocked this server's ip address
$this->cache->set($forbiddenKey, true, 60 * 61);
throw new RateLimitException();
throwRateLimitException();
} elseif ($e->getCode() === 429) {
$this->cache->set($rateLimitKey, true, 60 * 61);
throw new RateLimitException();
throwRateLimitException();
}
throw $e;
}

View File

@@ -355,7 +355,7 @@ class ReutersBridge extends BridgeAbstract
return $base_url . 'articles-by-section-alias-or-id-v1?query=' . $json_query;
break;
}
returnServerError('unsupported endpoint');
throwServerException('unsupported endpoint');
}
private function addStories($title, $content, $timestamp, $author, $url, $category)

View File

@@ -65,7 +65,7 @@ class RutubeBridge extends BridgeAbstract
private function getJSONData($html)
{
$jsonDataRegex = '/window.reduxState = (.*);/';
preg_match($jsonDataRegex, $html, $matches) or returnServerError('Could not find reduxState');
preg_match($jsonDataRegex, $html, $matches) or throwServerException('Could not find reduxState');
$map = [
'\x26' => '&',
'\x3c' => '<',

View File

@@ -20,7 +20,7 @@ class SIMARBridge extends BridgeAbstract
{
$html = getSimpleHTMLDOM($this->getURI());
$e_home = $html->find('#home', 0)
or returnServerError('Invalid site structure');
or throwServerException('Invalid site structure');
foreach ($e_home->find('span') as $element) {
$item = [];
@@ -34,7 +34,7 @@ class SIMARBridge extends BridgeAbstract
if ($this->getInput('interventions')) {
$e_main1 = $html->find('#menu1', 0)
or returnServerError('Invalid site structure');
or throwServerException('Invalid site structure');
foreach ($e_main1->find('a') as $element) {
$item = [];

View File

@@ -0,0 +1,55 @@
<?php
declare(strict_types=1);
class SamMobileUpdateBridge extends BridgeAbstract
{
const NAME = 'SamMobile updates';
// pull info from this site
const URI = 'https://www.sammobile.com/samsung/security/';
const DESCRIPTION = 'Fetches the latest security patches for Samsung devices';
const MAINTAINER = 'floviolleau';
const PARAMETERS = [
[
'model' => [
'name' => 'Model',
'exampleValue' => 'SM-S926B',
'required' => true,
],
'country' => [
'name' => 'Country',
'exampleValue' => 'EUX',
'required' => true,
]
]
];
const CACHE_TIMEOUT = 7200; // 2h
public function collectData()
{
$model = $this->getInput('model');
$country = $this->getInput('country');
$uri = self::URI . $model . '/' . $country;
$html = getSimpleHTMLDOM($uri);
$elementsDom = $html->find('.main-content-item__content.main-content-item__content-md table tbody tr');
foreach ($elementsDom as $elementDom) {
$item = [];
$td = $elementDom->find('td');
$title = 'Security patch: ' . $td[2] . ' - Android version: ' . $td[3] . ' - PDA: ' . $td[4];
$text = 'Model: ' . $td[0] . '<br>Country/Carrier: ' . $td[1] . '<br>Security patch: ' . $td[2] . '<br>OS version: Android ' . $td[3] . '<br>PDA: ' . $td[4];
$item['uri'] = $uri;
$item['title'] = $title;
$item['author'] = self::MAINTAINER;
$item['timestamp'] = (new DateTime($td[2]->innertext))->getTimestamp();
$item['content'] = $text;
$item['uid'] = hash('sha256', $item['title']);
$this->items[] = $item;
}
}
}

View File

@@ -59,7 +59,7 @@ class SchweinfurtBuergerinformationenBridge extends BridgeAbstract
if (preg_match('/artikel_id_(\d+)/', $article->id, $match)) {
$articleIDs[] = $match[1];
} else {
returnServerError('Couldn\'t determine article ID from index page.');
throwServerException('Couldn\'t determine article ID from index page.');
}
}

View File

@@ -71,7 +71,7 @@ EOD;
{
if (!is_null($this->getInput('profile'))) {
preg_match($this->profileUrlRegex, $this->getInput('profile'), $user)
or returnServerError('Could not extract user ID and name from given profile URL.');
or throwServerException('Could not extract user ID and name from given profile URL.');
return self::URI . '/' . $user[1] . '/uploads';
}

View File

@@ -67,7 +67,7 @@ class SensCritiqueBridge extends BridgeAbstract
private function extractDataFromList($list)
{
if ($list === null) {
returnClientError('Cannot extract data from list');
throwClientException('Cannot extract data from list');
}
foreach ($list->find('div[data-testid="product-list-item"]') as $movie) {

View File

@@ -48,20 +48,20 @@ class SeznamZpravyBridge extends BridgeAbstract
$html = getSimpleHTMLDOMCached($url . $this->getInput('author'), $ONE_DAY);
$mainBreadcrumbs = $html->find($selectors['breadcrumbs'], 0)
or returnServerError('Could not get breadcrumbs for: ' . $this->getURI());
or throwServerException('Could not get breadcrumbs for: ' . $this->getURI());
$author = $mainBreadcrumbs->last_child()->plaintext
or returnServerError('Could not get author for: ' . $this->getURI());
or throwServerException('Could not get author for: ' . $this->getURI());
$this->feedName = $author . ' - Seznam Zprávy';
$articles = $html->find($selectors['articleList'])
or returnServerError('Could not find articles for: ' . $this->getURI());
or throwServerException('Could not find articles for: ' . $this->getURI());
foreach ($articles as $article) {
// Get article URL
$titleLink = $article->find($selectors['articleTitle'], 0)
or returnServerError('Could not find title for: ' . $this->getURI());
or throwServerException('Could not find title for: ' . $this->getURI());
$articleURL = $titleLink->href;
$articleContentHTML = getSimpleHTMLDOMCached($articleURL, $ONE_DAY);
@@ -71,9 +71,9 @@ class SeznamZpravyBridge extends BridgeAbstract
// Article text content
$contentElem = $articleContentHTML->find($selectors['articleContent'], 0)
or returnServerError('Could not get article content for: ' . $articleURL);
or throwServerException('Could not get article content for: ' . $articleURL);
$contentParagraphs = $contentElem->find($selectors['articleParagraphs'])
or returnServerError('Could not find paragraphs for: ' . $articleURL);
or throwServerException('Could not find paragraphs for: ' . $articleURL);
// If the article has an image, put that image at the start
$contentInitialValue = isset($articleImageElem) ? $articleImageElem->outertext : '';
@@ -83,7 +83,7 @@ class SeznamZpravyBridge extends BridgeAbstract
// Article categories
$breadcrumbsElem = $articleContentHTML->find($selectors['breadcrumbs'], 0)
or returnServerError('Could not find breadcrumbs for: ' . $articleURL);
or throwServerException('Could not find breadcrumbs for: ' . $articleURL);
$breadcrumbs = $breadcrumbsElem->children();
$numBreadcrumbs = count($breadcrumbs);
$categories = [];
@@ -96,7 +96,7 @@ class SeznamZpravyBridge extends BridgeAbstract
// Article date & time
$articleTimeElem = $article->find($selectors['articleTime'], 0)
or returnServerError('Could not find article time for: ' . $articleURL);
or throwServerException('Could not find article time for: ' . $articleURL);
$articleTime = $articleTimeElem->plaintext;
$articleDMElem = $article->find($selectors['articleDM'], 0);

View File

@@ -41,7 +41,7 @@ class ShanaprojectBridge extends BridgeAbstract
$html = $this->loadSeasonAnimeList();
$animes = $html->find('div.header_display_box_info')
or returnServerError('Could not find anime headers!');
or throwServerException('Could not find anime headers!');
$min_episodes = $this->getInput('min_episodes') ?: 0;
$min_total_episodes = $this->getInput('min_total_episodes') ?: 0;
@@ -89,7 +89,7 @@ class ShanaprojectBridge extends BridgeAbstract
$html = defaultLinkTo($html, self::URI . '/seasons');
$season = $html->find('div.follows_menu > a', 1)
or returnServerError('Could not find \'Season Anime List\'!');
or throwServerException('Could not find \'Season Anime List\'!');
$html = getSimpleHTMLDOM($season->href);
@@ -104,7 +104,7 @@ class ShanaprojectBridge extends BridgeAbstract
private function extractAnimeTitle($anime)
{
$title = $anime->find('a', 0)
or returnServerError('Could not find anime title!');
or throwServerException('Could not find anime title!');
return trim($title->innertext);
}
@@ -112,7 +112,7 @@ class ShanaprojectBridge extends BridgeAbstract
private function extractAnimeUri($anime)
{
$uri = $anime->find('a', 0)
or returnServerError('Could not find anime URI!');
or throwServerException('Could not find anime URI!');
return $uri->href;
}
@@ -144,7 +144,7 @@ class ShanaprojectBridge extends BridgeAbstract
private function extractAnimeEpisodeInformation($anime)
{
$episode = $anime->find('div.header_info_episode', 0)
or returnServerError('Could not find anime episode information!');
or throwServerException('Could not find anime episode information!');
$retVal = preg_replace('/\r|\n/', ' ', $episode->plaintext);
$retVal = preg_replace('/\s+/', ' ', $retVal);
@@ -162,7 +162,7 @@ class ShanaprojectBridge extends BridgeAbstract
return $matches[1];
}
returnServerError('Could not extract background image!');
throwServerException('Could not extract background image!');
}
// Builds an URI to search for a specific anime (subber is left empty)

View File

@@ -85,7 +85,7 @@ class SitemapBridge extends CssSelectorBridge
$links = $this->sitemapXmlToList($sitemap_xml, $url_pattern, empty($limit) ? 10 : $limit);
if (empty($links) && empty($this->sitemapXmlToList($sitemap_xml))) {
returnClientError('Could not retrieve URLs with Timestamps from Sitemap: ' . $sitemap_url);
throwClientException('Could not retrieve URLs with Timestamps from Sitemap: ' . $sitemap_url);
}
foreach ($links as $link) {
@@ -117,7 +117,7 @@ class SitemapBridge extends CssSelectorBridge
$url = urljoin($url, '/sitemap.xml');
return $sitemap;
} else {
returnClientError('Failed to locate Sitemap from /robots.txt or /sitemap.xml. Try setting it manually.');
throwClientException('Failed to locate Sitemap from /robots.txt or /sitemap.xml. Try setting it manually.');
}
}
$url = $matches[1];

View File

@@ -562,7 +562,7 @@ class SkimfeedBridge extends BridgeAbstract
private function extractFeed($html, $author)
{
$articles = $html->find('li')
or returnServerError('Could not find articles!');
or throwServerException('Could not find articles!');
if (
count($articles) === 1
@@ -575,7 +575,7 @@ class SkimfeedBridge extends BridgeAbstract
foreach ($articles as $article) {
$anchor = $article->find('a', 0)
or returnServerError('Could not find anchor!');
or throwServerException('Could not find anchor!');
$item = [];
@@ -600,13 +600,13 @@ class SkimfeedBridge extends BridgeAbstract
private function extractHotTopics($html)
{
$topics = $html->find('#popbox ul li')
or returnServerError('Could not find topics!');
or throwServerException('Could not find topics!');
$limit = $this->getInput('limit') ?: -1;
foreach ($topics as $topic) {
$anchor = $topic->find('a', 0)
or returnServerError('Could not find anchor!');
or throwServerException('Could not find anchor!');
$item = [];
@@ -624,11 +624,11 @@ class SkimfeedBridge extends BridgeAbstract
private function extractCustomFeed($html)
{
$boxes = $html->find('#boxx .boxes')
or returnServerError('Could not find boxes!');
or throwServerException('Could not find boxes!');
foreach ($boxes as $box) {
$anchor = $box->find('span.boxtitles a', 0)
or returnServerError('Could not find box anchor!');
or throwServerException('Could not find box anchor!');
$author = '<a href="' . $anchor->href . '">' . trim($anchor->plaintext) . '</a>';
$uri = $anchor->href;
@@ -667,11 +667,11 @@ class SkimfeedBridge extends BridgeAbstract
$html = getSimpleHTMLDOMCached(static::URI);
if (!$this->isCompatible($html)) {
returnServerError('Skimfeed version is not compatible!');
throwServerException('Skimfeed version is not compatible!');
}
$boxes = $html->find('#boxx .boxes')
or returnServerError('Could not find boxes!');
or throwServerException('Could not find boxes!');
// begin of 'channel' list
$message = <<<EOD
@@ -686,7 +686,7 @@ EOD;
foreach ($boxes as $box) {
$anchor = $box->find('span.boxtitles a', 0)
or returnServerError('Could not find box anchor!');
or throwServerException('Could not find box anchor!');
$title = trim($anchor->plaintext);
$uri = $anchor->href;
@@ -723,11 +723,11 @@ EOD;
$html = getSimpleHTMLDOMCached(static::URI);
if (!$this->isCompatible($html)) {
returnServerError('Skimfeed version is not compatible!');
throwServerException('Skimfeed version is not compatible!');
}
$channels = $html->find('#menubar a')
or returnServerError('Could not find channels!');
or throwServerException('Could not find channels!');
// begin of 'tech_channel' list
$message = <<<EOD
@@ -759,11 +759,11 @@ EOD;
$channel_html = getSimpleHTMLDOMCached(static::URI . $uri);
$boxes = $channel_html->find('#boxx .boxes')
or returnServerError('Could not find boxes!');
or throwServerException('Could not find boxes!');
foreach ($boxes as $box) {
$anchor = $box->find('span.boxtitles a', 0)
or returnServerError('Could not find box anchor!');
or throwServerException('Could not find box anchor!');
$boxtitle = trim($anchor->plaintext);
$boxuri = $anchor->href;

Some files were not shown because too many files have changed in this diff Show More