Anidex uses two separate domains, anidex.info and anidex.moe
anidex.info has ddos-guard so we need to request
anidex.moe with Host header set to anidex.info
Separate bridge from Releases3DS that just has a different URL.
Inherits from Releases3DS so both bridges need to be present.
* [Releases3DS] Fix PHP notices related to IGN
* [DarkReading] Hide dummy articles
* [FuturaSciences] Strip inline scripts from content
* [FeedExpander] Fix PHP notice on missing uri field
(guid is valid uri AND item uri is not valid)
=> (guid is valid uri AND item uri is empty or not valid)
* [NextInpact] Fix subtitle extraction
* [Markdown] Fix images with empty replacement text
* [TheHackerNews] Fix Author name cleanup
* [LeMondeInformatique] Remove encoding conversion
Was previously needed due to actual encoding on the page
being inconsistent with encoding specified in <meta> tag
* [AnimeUltime] Remove encoding conversion
Was previously needed due to encoding on the page being incorrect
* [FuturaSciences] Fix content extraction
* [FuturaSciences] Fix unneeded unset()
* [GBAtemp] Fix tutorial mode URL extraction
* [GBAtemp] Fix tutorial mode Title extraction
- Adapted from https://stackoverflow.com/a/41575677
- Tested by running `php -m`
- SimpleXML now comes pre-installed with PHP, so removed the extra step
- Adds --no-install-recommends
Signed-off-by: Nemo <me@captnemo.in>
Twitter has changed URL scheme back again (see PR#1647 / commit 78298385d0)
This patch will try both URL schemes now and throw a specific error when neither works
* [DribbbleBridge] Fixed regular expressions for quote replacement in JSON (previously invalid JSON was created if a property value contained colons or single quotes). Also updated two CSS selectors as Dribbble's HTML has changed.
* [DribbbleBridge] Added fix for relative dates in JSON
* [DribbbleBridge] Removed redundant whitespaces
* [TwitterBridge] guest token is returned via body again. This change will try to search fot token inside header and fallback to body
* Twitter changed the URL scheme for the API
* [TwitterBridge] Expire guest token by time
In addition to fetching a new guest token after 100 uses, also expire token after 5 minutes (configurable).
- Remove movies search. It appears the website changed their movies
displays and data cannot be easily extracted for now.
- Fix some errors on items without proper description and/or original
title.
An official RSS feed is available at https://blog.revolut.com/rss/
Note that there is also an invisible "RSS" button next to the Facebook
and Twitter icons at the menu bar.
References #1321
Facebook has changed their strategy regarding permalinks, which
now include lots of unnecessary target data. Fortunately it also
contains the unique story id which we can utilize as URI.
* [InstagramBridge] Fix GraphSidecar output
Fix following issues which related to output of the GraphSidecar type posts.
- The GraphSidecar post's media wasn't outputted except for first picture when searching by hashtag or location
- Video didn't embedded
NOTE:
The function getInstagramStory() which was called when the post type is GraphSidecar didn't seem to work just as one intended.
Because the web request called in that function is just to get the media of single post, NOT to get the media of Story.
But I don't have any idea to solve #694, so it seems be better to rename these function and member variable properly.
Adds <link> elements for each additional output format in the <head> of
HTML format output to allow RSS readers to find the actual feeds
directly from the HTML page.
This bridge allows you to follow iOS/iPad application updates
directly over RSS. This allows you to get notified of an application
update even if you don't have a iDevice. This may be useful in cases
where you want to be notified of a competitor's application for eg.
I built this after I got tired of waiting for WhatsApp to push their
security release on iOS. It shows up on the AppStore 7 days later
Most of the code in RSS-Bridge uses the long array syntax.
This commit adds a check to enforce using this syntax over
the short array syntax.
All failures have been fixed.
Debian increased the minimum security level for OpenSSL from TLS 1.0
to TLS 1.2 [1] which also affects the Debian-based PHP image for Docker.
This change can break some bridges which have to connect to servers with
lower security level. Since all browsers still connect to these servers,
so should RSS-Bridge.
Note that according to [2] Mozilla, Firefox, Microsoft, Google and Apple
plan to increase the minimum security level to TLS 1.2 around March 2020.
At this time RSS-Bridge should follow the browser changes.
This commit updates the Dockerfile to automatically drop the minimum
security level back to TLS 1.0.
Based on the solution provided by @theScrabi in #1318
[1] https://wiki.debian.org/ContinuousIntegration/TriagingTips/openssl-1.1.1
[2] 553fc8e61f/debian/libssl1.1.NEWS
setInputs() currently looks if the global array defines a 'value'
for a given parameter, but that isn't supported by the API. It
needs to be 'defaultValue'.
This bridge currently takes a very long time to process
all news items on the page, when in many cases only one
or two had been added since the last check.
This commit adds a new parameter 'limit', which defines
the maximum number of items to add to the feed. This is
an optional paramter that defaults to 3.
This bridge currently takes a very long time to process
all news items on the page, when in many cases only one
or two had been added since the last check.
This commit adds a new parameter 'limit', which defines
the maximum number of items to add to the feed. This is
an optional paramter that defaults to 3.
* action: Add action to check bridge connectivity
It is currently not simply possible to check if the remote
server for a bridge is reachable or not, which means some
of the bridges might no longer work because the server is
no longer on the internet.
In order to find those bridges we can either check each
bridge individually (which takes a lot of effort), or use
an automated script to do this for us.
If a server is no longer reachable it could mean that it is
temporarily unavailable, or shutdown permanently. The results
of this script will at least help identifying such servers.
* [Connectivity] Use Bootstrap container to properly display contents
* [Connectivity] Limit connectivity checks to debug mode
Connectivity checks take a long time to execute and can require a lot
of bandwidth. Therefore, administrators should be able to determine
when and who is able to utilize this action. The best way to prevent
regular users from accessing this action is by making it available in
debug mode only (public servers should never run in debug mode anyway).
* [Connectivity] Split implemenation into multiple files
* [Connectivity] Make web page responsive to user input
* [Connectivity] Make status message sticky
* [Connectivity] Add icon to the status message
* [contents] Add the ability for getContents to return header information
* [Connectivity] Add header information to the reply Json data
* [Connectivity] Add new status (blue) for redirected sites
Also adds titles to status icons (Successful, Redirected, Inactive, Failed)
* [Connectivity] Fix show doesn't work for inactive bridges
* [Connectivity] Fix typo
* [Connectivity] Catch errors in promise chains
* [Connectivity] Allow search by status and update dynamically
* [Connectivity] Add a progress bar
* [Connectivity] Use bridge factory
* [Connectivity] Import Bootstrap v4.3.1 CSS
The current solution for titles on input boxes is not obvious to the
user as support varies between bridges. This commit adds an button to
all input boxes with titles in order to make it clear to the user that
additional information is available.
Allows getting the expected MIME type of the format's output. A
corresponding MIME_TYPE constant is also defined in FormatAbstract for
the format implementations to overwrite.
Error reporting currently takes place for each error. This can result
in many error messages if a server has connectivity issues (i.e. when
it re-connects to the internet every 24 hours).
This commit adds a new option to the configuration file to define the
number of error reports to suppress before returning an error message
to the user.
Error reports are cached and therefore automatically purged after 24
hours. A successful bridge request does **not** clear the error count
as sporadic issues can be the result of actual problems on the server.
The implementation currently makes no assumption on the type of error,
which means it also suppresses bridge errors in debug mode. The default
value is, however, set to 1 which means all errors are reported.
References #994
Bridge errors are currently included as part of the feed to
notify users about erroneous bridges (before that, bridges
silently failed).
This solution, however, can produce a high load of error
messages if servers are down (see #994 for more details).
Admins may also not want to include error messages in feeds
in order to keep those kind of problems away from users or
simply to silently fail by choice.
This commit adds a new configuration section "error" with
one option "output" which can be set to following values:
"feed": To include error messages in the feed (default)
"http": To return a HTTP header for each error
"none": To disable error reporting
Note that errors are always logged to 'error.log' independent
of the settings above.
Closes#1066
The composer file currently lacks a lot of details, especially the
"name" and "description", but also "require-dev" and "suggest" info.
This commit adds many more details to the composer file and updates
composer.lock for this repository. Technically the project is ready
to be shipped as composer package.
* Clarify Facebook bridges status
Distinguish between both Facebook bridges by their title.
This preserves all existing URLs.
* Update all URLs to secure HTTPS versions.
* Configure author name abbreviation
* Improve feed names
Use the correct feed name on each bridge.
Make sure the feed names don't repeat the "Facebook" name.
* [ParameterValidator] Ensure context has all fields
Previously if a bridge had a set of parameters like:
const PARAMETERS = array(
'ContextA' => array(
'Param1' => array(
'name' => 'Param1',
'required' => true
)
),
'ContextB' => array(
'Param1' => array(
'name' => 'Param1',
'required' => true
),
'Param2' => array(
'name' => 'Param2',
'required' => true
)
)
)
and a query specifying both Param1 and Param2 was provided a 'Mixed
context parameters' error would be returned. This change ensures
ContextA in the above example would not be considered a relevant context.
* [BandcampBridge] Add band and artist feeds
This can return a limited number of the most recent releases by a band,
or a single release/album. Each release may be given a unique article ID
depending on its track list with the "Releases, new one when track track
changes" option, which should make them show up as new articles when
tracks are added or removed. Releases may also be split up to individual
articles for each track with the "Individual tracks" option.
This uses and undocumented API from the Bandcamp Android app. It's much
faster than loading and parsing the website HTML, and seems to fail less
often with more relaxed rate limits. It's still far from perfect in that
regard though.
The "Individual tracks" option generates requests for each individual
track so that can quickly run into rate limits.
The "Individual tracks" option also has a quirk where tracks released
under e.g. a music label will have their artist set to the label instead
of the actual artist of the track. This is a limitation of the API.
* [Rule34pahealBridge] Use HTTPS
* [KonachanBridge] Use HTTPS
* [Rule34Bridge] Use HTTPS
* [SafebooruBridge] Use HTTPS
* [TbibBridge] Use HTTPS
* [XbooruBridge] Use HTTPS
* [ScmbBridge] Use HTTPS
* [ReporterreBridge] Use HTTPS
* [BastaBridge] Use HTTPS
* [NiceMatinBridge] Use HTTPS
* [ScoopItBridge] Use HTTPS
* [TheCodingLoveBridge] Use HTTPS
* [Shimmie2Bridge] Use HTTPS
* [HDWallpapersBridge] Use HTTPS
* [GiphyBridge] Use HTTPS
* [PickyWallpapersBridge] Use HTTPS
* [ParuVenduImmoBridge] Use HTTPS
* [ElsevierBridge] Use HTTPS
* [CastorusBridge] Use HTTPS
* [CollegeDeFranceBridge] Use HTTPS
* [MangareaderBridge] Use HTTPS
* [FacebookBridge] Prevent sending empty header
When running in CLI mode, `getEnv('HTTP_ACCEPT_LANGUAGE')` returns `false`. In that case, don't send the `Accept-Language` header.
* [PatreonBridge] Add new bridge
* [PatreonBridge] Add UID to articles
Patreon changes post URLs when the post title is updated, so set a UID
based on the post ID instead.
The Website changed in two way :
- The filter about availability disappeared (and this leads to a
parameters change, which will break existing bridges, sorry)
- Some HTML change
Directly accessing ...->href resulted in a string that contained '&'
instead of '&'. This was later escaped again to '&amp;' in some
formats (e.g. Atom).
Removes 'self::URI' from processUpload() which was creating malformed URLs. Relative URLs are handled by defaultLinkTo() making 'self::URI' unnecessary.
* [TwitterBridge] Get cookies before sending request
Twitter now requires cookies to be set before requesting a page. This will fetch the cookies and send them to `getSimpleHTMLDOM()`.
* Formatting fixes
The public service demonoid.pw is no longer available and is
currently being rebuild under demonoid.info which hides torrents
behind a login wall. As this is not supported by RSS-Bridge, the
bridge will be removed.
Find more details on Reddit:
https://www.reddit.com/r/Demonoid/
* Remove getSearchResults()
* Remove ''From search' from PARAMETERS array
* Update getURI() and getName()
* Update collectData()
* Add '.rss' to URL in `collectData` instead of in `getURI`
Implements a bridge for
https://www.indeed.com/ (or any of the local variants)
Features:
- Takes a company name and returns a list of reviews and comments
- Limit the maximum number of items to return (default: 20)
- No upper limit on the number of items to return
- Search by language code (45 options)
- Supports detectParameters for any supported URL
Adds styles for display sizes smaller than 768px where
elements are currently hardly usable. Note that RSS-Bridge
is not designed for mobile use, but some users may want
to try things on their mobile phone before using it in
real life applications.
Resolves#796
RSS-Bridge currently has to guess the queried context from the data
provided by the user. This, however, can cause issues for bridges
that have multiple contexts with conflicting parameters (i.e. none).
This commit adds context hinting to queries via '&context=<context>'
which can be omitted in which case the context is determined as before.
Adding or removing feed formats from the "formats/" directory
currently has no effect on the buttons shown in the HTML format.
This can cause errors if users press one of the buttons for a
format that is no longer available on the server.
This commit changes the behavior to dynamically add buttons based
on the available formats. Syndication feeds, however, are no longer
supported as they require knowledge about the content type, which
is not known without further changes to the formats API (may be
added later if there is a demand).
Closes#942
The format factory can be based on the abstract factory class if it
wasn't static. This allows for higher abstraction and makes future
extensions possible. Also, not all parts of RSS-Bridge need to work
on the same instance of the factory.
References #1001
The cache factory can be based on the abstract factory class if it
wasn't static. This allows for higher abstraction and makes future
extensions possible. Also, not all parts of RSS-Bridge need to work
on the same instance of the factory.
References #1001
The bridge factory can be based on the abstract factory class if it
wasn't static. This allows for higher abstraction and makes future
extensions possible. Also, not all parts of RSS-Bridge need to work
on the same instance of the bridge factory.
References #1001
RSS-Bridge currently sanitizes the format name only for the display
action, which can cause problems if other actions depend on formats
as well.
It is therefore better to do sanitization in the factory class for
formats. Additionally, formats should not require a perfect match,
so 'Atom' and 'aToM' make no difference. This will also allow users
to define formats in their own style (i.e. only lowercase via CLI).
References #1001
Heroku requires the file `app.json` as well as the composer files
`composer.json` and `composer.lock` to deploy a service. Deploy
doesn't work if these files are ignored during export (because of
the way this service deploys projects).
This commit adds comments to .gitattributes to prevent this issue
from re-appearing in the future. All affected lines are commented
out.
Also added some spacing for better readability.
References #1165
detectParameters() is called in a loop for all bridges on a URL, thus
if a bridge returns an error message, the output messages get mixed
up and all detect operations fail.
This seems to be a limitation of the detect function for now.
The target site for this bridge has been down for at least a year
now and there doesn't seem to be any attempt to get it back up.
Their twitter account is also silent since 2012, so no harm
removing this bridge.
https://twitter.com/dollbooru?lang=en
- Add function to build an URL to the GitHub issue comment
- Change scope of internal functions from protected to private
- Use IDs instead of classes as comment selectors, to include the
issue author in the output feed.
References #1100
Response headers may contain fields with no values.
Example:
"Referrer-Policy: "
In this case the current implementation of explode() results in an
error because there is no content after ": ". Changing the delimiter
to ":" and trimming the value manually fixes that issue.
Users currently only get one option: to open a new issue on GitHub.
This can, however, result in duplicate issues, which is not desired.
This commit adds a second button to the error message, which links
to the GitHub issues tracker with the search query set to find
errors for the current bridge. That way, users can collaborate
on the same issue.
The placeholder is currently visible on key focus and only hidden
once a user starts typing. This can be confusing and doesn't look
good.
As it turns out, ::placeholder is an official selector:
https://developer.mozilla.org/en-US/docs/Web/CSS/::placeholder
For some reason, listing placeholder selectors with "," doesn't
work on some browsers (tested in FF 60 ESR). Making each of the
selectors explicit works, however.
Incorrect configuration values are currently handled individually
for each condition, resulting in a lot of repetitive operations.
This commit adds two new private functions to report errors to the
user and end execution of the script.
The configuration files are currently hard-coded in the configuration
classes and error messages. However, the implementation should not
rely on specific details like the file name. Instead, the files should
be part of the global definition.
This commit introduces two global constants for the configuration files
- FILE_CONFIG => 'config.ini.php'
- FILE_CONFIG_DEFAULT => 'config.default.ini.php'
RSS-Bridge currently statically sets the timezone to UTC which can
result in incorrect timestamps if the server is hosted in another
region.
This commit adds a new configuration parameter to allow admins to
specify their own timezone for their servers. Invalid values will
result in an error message.
Example:
[system]
timezone = "UTC"
For compatibility reasons the default value is set to UTC.
This parameter accepts any of the supported timezones listed at
https://www.php.net/manual/en/timezones.phpCloses#956
References #1001
If the bridge name matches exactly, it is not necessary to perform
a strtolower compare of bridges. In some situations this can lead
to much faster response times (depending on the amount of bridges
in whitelist.txt).
Default bridges are currently statically defined in index.php, which
is not the right place if we want to keep responsibilities separated.
This commit introduces a new file whitelist.default.txt that holds
the default bridges and which is loaded automatically, if whitelist.txt
doesn't exist.
Due to this it is also no longer necessary to have write permission
for the root directory.
References #1001
This reverts commit 052844f5e1.
There is a bug in ->remove() that causes the parser to incorrectly
identify elements in the DOM tree that shouldn't exist anymore.
References #1151
simplehtmldom 1.9 introduced new functions to recursively remove
nodes from the DOM. This allows removing elements without the need
to re-load the document by using $html->load($html->save()), which
is very inefficient.
Find more information about remove() at
https://simplehtmldom.sourceforge.io/docs/1.9/api/simple_html_dom_node/remove/
find('*') wasn't supported in older versions of simplehtmldom but it
is supported now. Thus, all custom implementations can be replaced
by the correct solution.
This fixes the following issue:
1. bridge sets unique ids for the items (ids get hashed)
2. items go to the cache
3. on next run items get loaded from cache
4. these items have different ids because they were hashed again
5. they show up twice in feed reader
* Switch Docker Image to official php base image
Switch from the unofficial Alpine+php image to the official php-apache image.
This has 2 advantages:
1. Official image is guaranteed to have regular updates, etc
2. The persistent Docker Alpine DNS Issue goes away;
https://github.com/gliderlabs/docker-alpine/issues/255
* [Docker] Ignore more files from Docker Image
- For consistency, functions should always return null on non-existing data.
- WordPressPluginUpdateBridge appears to have used its own cache instance in the past. Obviously not used anymore.
- Since $key can be anything, the cache implementation must ensure to assign the related data reliably; most commonly by serializing and hashing the key in an appropriate way.
- Even though the default path for storage is perfectly fine, some people may want to use a different location. This is an example how a cache implementation is responsible for its requirements.
* [BingSearch] Make the bridge compatible with PHP 5.6
The use of isset() with an expression is not possible in PHP 5.6. I
fixed it by replacing isset() with "null !== ".
This seems like a weird bug somewhere.
Either the HTML parser should return the valid page, or the CMS should
not convert the URL first, or the URL validation regex is buggy.
* [AutoJMBridge] Fix bridge after website change
The website was totally reworked, so the bridge had to be reworked too.
The bridge parameters changed, therefore old RSS feed will not work
anymore, but it was impossible to do it in another way.
PHP nightly recently got updated to dev-8.x, which is not supported
by any of the test scripts. This makes the test pretty useless and
doesn't help in any way.
Instead, the build script should focus on current versions of PHP,
starting from 5.6 to 7.3 (current stable release).
PHP 7.3 is a reasonable version to use for finding breaking changes
in the test scripts (phpunit especially warns about changes). These
tests can fail, of course.
In case of a unparsable date, the text to DateTime object failed, and
this resulted to a Fatal error while using this DateTime object . To
prvent this fatal error, if the date parsing failse, then a DateTime
object is created with the actual date.
* rework to parse new layout
* skip incomplete rows
The last row could have fewer columns if there are less rows than the items limit. This usually should not happen, though.
* use constant for skipping
Bandcamp tags pages have a new layout and now use a POST API endpoint to view each page of releases.
Output of this bridge should be almost the same as before, with a few small improvements:
- Small album image in 'content', larger album image in 'enclosures'
- RSS item titles/authors are appended with the releaser in parentheses if the artist name and the releaser are different (i.e. Record Label's Bandcamp releases an album called Bar by the band named Foo, it would get the title 'Foo - Bar (Record Label)' and the author 'Foo (Record Label)')
* Fix issue with CachetAPI Pagination
Fixing issue that only the oldest 20 entries were shown.
_Background:_
_Cachet has a, lets call it odd, system of pagination. On the first page you see the incidents first created, so they are not what you want to see. But on the last page you can have 1 or 20 of the newest incidents. So you have to take the incidents from the last page (call it Pmax) and combine them with the incidents from Pmax - 1._
This commit adds support for a new parameter which specifies the type
of cache to use for caching. It is specified in config.ini.php:
[cache]
type = "..."
Currently only one type of cache is supported (see /caches). All uses
of 'FileCache' were replaced by this configuration option.
Note: Caching currently depends on files and folders (due to FileCache).
Experience may vary depending on the selected cache type. For now always
check if FileCache is working before testing alternative types.
References #1000
'uid' represents the unique id for a feed item. This item is null by
default and can be set to any string value. The provided string value
is always hashed to sha1 to make it the same length in all cases.
References #977, #1005
Fix PHP Notice: Undefined offset: 0. Error below triggers when there are no items:
PHP Notice: Undefined offset: 0 in C:\php\rss-bridge\index.php on line 249
This commit makes Travis fail on deprecation warnings for new versions
of PHP and ensures that checks are made against all versions from PHP
5.6 onwards
Add transformation from legacy items to FeedItems, before transforming
items to the desired format. This allows using legacy bridges alongside
bridges that return FeedItems.
As discussed in #940, instead of throwing exceptions on invalid
parameters, add messages to the debug log instead
Add support for strings to setTimestamp(). If the provided timestamp
is a string, automatically try to parse it using strtotime().
This allows bridges to simply use `$item['timestamp'] = $timestamp;`
instead of `$item['timestamp'] = strtotime($timestamp);`
Support simple_html_dom_node as input paramter for setURI
Support simple_html_dom_node as input parameter for setContent
Squiz.WhiteSpace.SuperfluousWhitespace has problems detecting blank
lines in functions when used together with the PSR2 standard.
More information: https://github.com/squizlabs/PHP_CodeSniffer/issues/600
This commit fixes that issue by restoring the original behavior.
It also adds rules for function spacing because the sniff mentioned
above does only work within functions.
In item content, the image is now a clickable link to the post;
In item title a ▶ is prepended if the post contains a video; it's impossible to tell from the content image.
Images in Twitter feeds are currently being scaled by adding ':orig'
(original image) and ':thumb' (thumbnail) to image URIs in the feed.
This can cause issues with feed readers that don't handle colons in
URIs correctly.
Image scaling can now be disabled by adding '&noimgscaling=on' to the
query. This parameter is optional to stay compatible to existing feeds.
References #957
- Update parser to version 1.7
https://sourceforge.net/projects/simplehtmldom/files/simplehtmldom/1.7/
References #959
-------------------- CHANGELOG --------------------
- Added code documentation to improve readability
- Added unit tests for `simple_html_dom::$self_closing_tags`
- Added unit tests for `simple_html_dom::$optional_closing_tags`
- Added unit tests for bug reports
- Added test for bug [#56](https://sourceforge.net/p/simplehtmldom/bugs/56/)
- Added test for bug [#97](https://sourceforge.net/p/simplehtmldom/bugs/97/)
- Added test for bug [#116](https://sourceforge.net/p/simplehtmldom/bugs/116/)
- Added test for bug [#121](https://sourceforge.net/p/simplehtmldom/bugs/127/)
- Added test for bug [#127](https://sourceforge.net/p/simplehtmldom/bugs/127/)
- Added test for bug [#154](https://sourceforge.net/p/simplehtmldom/bugs/154/)
- Added test for bug [#160](https://sourceforge.net/p/simplehtmldom/bugs/160/)
- Added unit tests for memory management of the parser
- Added bit flags to `simple_html_dom::load()`
- Added bit flag `HDOM_SMARTY_AS_TEXT` to optionally filter Smarty scripts (#154)\
**Note**: Smarty scripts are no longer filtered by default!\
- Added build script to automate releases
- Added support for attributes without whitespace to separate them
- Improved documentation and readability for `$self_closing_tags`
- Improved documentation and readability for `$block_tags`
- Improved documentation and readability for `$optional_closing_tags`
- Updated list of `simple_html_dom::$self_closing_tags`
- Removed 'spacer' (obsolete)
- Added 'area'
- Added 'col'
- Added 'meta'
- Added 'param'
- Added 'source'
- Added 'track'
- Added 'wbr'
- Updated list of `simple_html_dom::$optional_closing_tags`
- Removed "nobr" (obsolete)
- Added 'th' as closable element to 'td'
- Added 'td' as closable element to 'th'
- Added 'optgroup' with 'optgroup' and 'option' as closable elements
- Added 'optgroup' as closable element to 'option'
- Added 'rp' with 'rp' and 'rt' as closable elements
- Added 'rt' with 'rt' and 'rp' as closable elements
- Clarified meaning of `simple_html_dom->parent`
- Changed default `$offset` for `file_get_html()` from -1 to 0 (#161)
- Changed `simple_html_dom::load()` to remove script tags before replacing newline characters
- `simple_html_dom_node::text()` no longer adds whitespace to top level span elements (only to sub-elements)
- `simple_html_dom_node::text()` adds blank lines between paragraphs
- Normalized line endings in the repository to LF via `.gitattributes`
- Improved performance of `simple_html_dom::parse_charset()` by approximately 25%
- Improved performance of `simple_html_dom::parse()` by approximately 10%
- `str_get_html()` is deprecated and should be replaced by `new simple_html_dom()`
- Removed protected function `simple_html_dom::copy_until_char_escaped()`
- Fixed compatibility issues with PHP 7.3
- Fixed typo (#147)
- Fixed handling of incorrectly escaped text (#160)
- Restore functionality of `$maxLen` in `file_get_html()`
- Fixed load_file breaks if an error ocurred in another script
1. Author Name now doesn't include Episode Title
2. It now fetches Episode Creation Timestamp, to allow correct sorting in podcatchers
3. Description is now the actual show notes, and not an <audio> tag
* core: Add bridge parameter auto detection
This adds a new 'detect' action which accepts a URL from which an
appropriate bridge is selected and relevant parameters are extracted.
The user is then automatically redirected to the selected bridge.
For example to get a feed from: https://twitter.com/search?q=%23rss-bridge
we could send a request to:
'/?action=detect&format=Atom&url=twitter.com/search%3Fq%3D%2523rss-bridge'
which would redirect to:
'/?action=display&q=%23rss-bridge&bridge=Twitter&format=Atom'.
This auto detection happens on a per-bridge basis, so a new function
'detectParameters' is added to BridgeInterface which bridges may implement.
It takes a URL for an argument and returns a list of parameters that were
extracted, or null if the URL isn't relevant for the bridge.
* [TwitterBridge] Add parameter auto detection
* [BridgeAbstract] Add generic parameter detection
This adds generic "paramater detection" for bridges that don't have any
parameters defined. If the queried URL matches the URI defined in the
bridge (ignoring https://, www. and trailing /) an emtpy list of parameters is
returned.
* [ZoneTelechargementBridge] Switch to the new Website
The website zone-telechargement1.org decided that he will be using a new
domain at the end of november :
https://www.annuaire-telechargement.com/
The bridge uses the new domain but still uses the same filename and
class name to keep the existing feed working.
Files with the option "export-ignore" are excluded from "git archive"
commands. Release files from GitHub will also ignore those files, so
packages are smaller and don't include unneccessary files.
This commit adds a cache for 'getContents' to '/cache/server'. All
contents are cached by default (even in debug mode). If debug mode
is enabled, the cached data is overwritten on each request.
In normal mode RSS-Bridge adds the 'If-Modified-Since' header with
the timestamp from the previously cached data (if available) to the
request.
Find more information on 'If-Modified-Since' here:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Modified-Since
If the server responds with "304 Not Modified", the cached data is
returned.
If the server responds with "200 OK", the received data is written
to the cache (creates a new cache file if it doesn't exist yet).
No changes were made for all other response codes.
Servers that don't support the 'If-Modified-Since' header, will
respond with "200 OK".
For servers that respond with "304 Not Modified", the required band-
width will decrease and RSS-Bridge will responding faster.
Files in the cache are forcefully removed after 24 hours.
Notice: Only few servers actually do support 'If-Modified-Since'.
Thus, most bridges won't be affected by this change.
simple_html_dom currently doesnt support "->find('*')", which is a
known issue: https://sourceforge.net/p/simplehtmldom/bugs/157/
The solution implemented by RSS-Bridge is to find all nodes WITHOUT
a specific attribute. If the attribute is very unlikely to appear
in the DOM, this is essentially returning all nodes.
This is the meaning behind
"->find('*[!b38fd2b1fe7f4747d6b1c1254ccd055e]')"
Error log reports "PHP Notice: Undefined offset: 2 in /rss-bridge/
lib/Debug.php on line 112" if the array returned by debug_backtrace
does not contain 3 items.
This commit fixes the issue by always using the last element in the
backtrace "end($backtrace)".
- Initialize with null to prevent leaking configurations
- Check if the working directory is a directory
- Store the real path instead of raw data
- Add final path separator as expected by Bridge::create
general
- Use self:: instead of Cache:: or static::
- Rename $dirCache to $workingDir
- Initialize $workingDir with null
function setDir
- Clear previous working directory before checking input parameters
- Change wording for the exception messages
- Store realpath instead of raw parameter
- Add path separator
This ensures the path always ends with the path separator, as assumed
by Cache::create
- Add check if the provided working directory is a valid directory
function getDir
- Use static parameter instead of function variable
- Change wording for the exception message
function create
- Rename parameter $nameCache to $name
- Rename $pathCache to $filePath
- Change wording for the exception messages
function isValidNameCache
- Rename function to isCacheName
- Rename parameter $nameCache to $name
- Explain in the function documentation the meaning of a 'valid name'
- Ensure Boolean return value (preg_match returns integer)
- Check if $name is a string
- Use slashes to enclose the regex
This is the first step in adding documentation to the core library
of RSS-Bridge. The documentation is not yet extracted by phpdoc,
yet may prove useful to anyone interested in starting with RSS-Bridge.
Commit e26d61e introduced a bug that causes the error message "The
bridge you [sic!] looking for does not exist." if the bridge name
specified in the query ends on "Bridge"
(i.e. '&bridge=SoundcloudBridge'), while other queries work fine
(i.e. '&bridge=Soundcloud').
This commit fixes that issue by sanitizing the bridge name before
creating the class.
References #922
Requesting `_cache_timeout` on servers where this option is disabled currently results in the error message 'This server doesn\'t support "_cache_timeout"!'. This commit changes that behavior to redirect to the query without `_cache_timeout`.
- Move all whitelisting functionality inside Bridge.php
- Set default whitelist once in index.php using Bridge::setWhitelist()
- Include bridge sanitizing inside Bridge.php
Bridge::sanitizeBridgeName($name)
Bridge.php now maintains the whitelist internally.
Also adds documentation to Debug.php!
* Debug::isEnabled()
Checks if the DEBUG file exists on disk on the first call (stored in
memory for the duration of the instance). Returns true if debug mode
is enabled for the client.
This function also sets the internal flag for Debug::isSecure()!
* Debug::isSecure()
Returns true if debuging is enabled for specific IP addresses, false
otherwise. This is checked on the first call of Debug::isEnabled().
If you call this function before Debug::isEnabled(), the default value
is false.
Replaces 'debugMessage' by specialized debug function 'Debug::log'.
This function takes the same arguments as the previous 'debugMessage'.
A separate Debug class allows for further optimization and separation
of concern.
- Remove HHVM
HHVM recently announced ending PHP support:
https://hhvm.com/blog/2018/09/12/end-of-php-support-future-of-hack.html
"HHVM v3.30 will be the last release series where HHVM aims to support
PHP. [...] Ultimately, we recommend that projects either migrate
entirely to the Hack language, or entirely to PHP7 and the PHP runtime."
RSS-Bridge never "officially" supported HHVM, so support can be removed.
- Use composer for all versions
PHP 5.6 is using PEAR, while all other versions use Composer to manage
packages and dependencies. This commit removes PEAR for PHP 5.6 in favor
of Composer. This also simplifies the script.
- Add PHP compatibility tests
Uses https://github.com/PHPCompatibility/PHPCompatibility
RSS-Bridge supports PHP 5.6 or higher. This commit adds tests to check
compatibility and detect breaking changes.
"phpcompatibility.xml" contains the ruleset.
Notice: Technically RSS-Bridge requires PHP 5.6.1, but for some reason
PHPCompatibility doesn't accept "5.6.1" for "testVersion". This is why
INI_SCANNER_TYPED is excluded from tests.
- Rearrange tests
PHP 5.6:
- Coding style (phpcs.xml)
- PHP compatibility (phpcompatibility.xml)
PHP 7.0:
- Coding style (phpcs.xml)
- Unit tests (phpunit.xml) - using stable release of PHPUnit
PHP nightly:
- Coding style (phpcs.xml)
- Unit tests (phpunit.xml) - using latest version of PHPUnit
- PHP compatibility - no exceptions for PHP 5.6+
- Documentation added to improve maintainability
This commit adds filters to remove embedded videos and view counts from
all posts. This doesn't remove the preview image for videos, which are
embedded separately.
Hidden elements are used for error conditions and generally made
visible using JavaScript. Since RSS-Bridge doesn't support JS, these
error messages are shown in the final feed. For example:
"It looks like you may be having problems playing this video. If so,
please try restarting your browser."
This commit removes all hidden elements to prevent error messages being
added to the feed.
- "It looks like you may be having problems playing this video. If so,
please try restarting your browser."
FB includes origin information (i.e. "YOUTUBE.COM") as well as
descriptions with embedded media (images and video).
These details are currently being removed by the bridge.
This commit changes implementation to only remove origin information
and keep the media description in place. The media description consists
of two elements - title and description. The title provided by FB is
included in an anchor, which gets replaced by a paragraph with the
same contents to improve readability.
References #912
- PATH_LIB_BRIDGES defines the path to bridges
- PATH_LIB_FORMATS defines the path to formats
- PATH_LIB_CACHES defines the path to caches
Include constants in RssBridge.php for consistency
Move CACHE_DIR from index.php to /lib/RssBridge.php and change name
to PATH_CACHE.
PATH_CACHE is one of the core paths of RSS-Bridge and should therefore
be defined in the core file RssBridge.php.
- Remove file documentation and license remark (defined in repository
scope - see README / UNLICENSE)
- Remove example usage (if necessary should be included in the Wiki)
"The require_once statement is identical to require except PHP will
check if the file has already been included, and if so, not include
(require) it again."
-- http://php.net/manual/en/function.require-once.php
Vendor files (simple_html_dom.php and urljoin.php) are included in the
repository and therefore shipped with all releases. If one of the files
is missing, either the repository or the release is incomplete.
PHP will generate error messages if either of the files is missing, so
there is no need to check availability manually unless it is done for
all files (which doesn't make sense because they are part of the
repository).
`defaultLinkTo` makes anchors point to the correct path which broke
parsing because it expected href to start with `#gistcomment`.
This commit changes the implementation to make `defaultLinkTo` point
to the correct page (using `getURI` instead of `self::URI`) and search
with `*=` instead of `^=`.
The parameter is used in error feeds. Since RSS-Bridge returns valid
feeds for error conditions, feed readers may attempt to access the
URI returned for the feed item in order to collect additional data,
thus including the parameter '_error_time' in the query.
This results in another error message, because it is an invalid input
parameter. Filtering the parameter allows RSS-Bridge to return the
original feed.
References #882
Instagram returns "HTTP/1.1 301 Moved Permanently" on each request
to "https://instagram.com/" because the correct location is
"https://www.instagram.com/".
Instagram will respond with "HTTP/1.1 301 Moved Permanently" if the
URI for the requested user doesn't end with a slash.
Notice: This is only a minor enhancement to prevent error 301 from
happening. The previous version worked fine as is.
Adds additional messages to the error log when fetching contents. The
data is helpful in finding issues with receiving contents from servers.
References: #879, #882, #884
Exceptions are reported to users, but they do not necessarily appear
in the error log on the server. Using 'error_log' we can explicitly
write exceptions and error messages to the log file, using the
standard PHP message format.
For more information see https://stackoverflow.com/a/26867035
Using the classical www.zone-telechargement1.org as base URL, the bridge will
always be redirected to the actual wwX.zone-telechargement1.org final URL. This
makes the bridge more robust to URL changes.
This commit collects the original contents from a different
tag to prevent this issue. The root cause is unknown but closely
related to the regex.
References #877
The function 'defaultLinkTo' applied to the source HTML does break
regex matches later in the bridge. We need to apply the function
right before adding the contents to the item for the bridge to work
properly.
References #856
- Base URL updated
- Show name has different styles on the Website, use another way to get the show name
- Entry URIs are now unique to make sure RSS readers don't treat episodes as duplicates
- No more new lines in the feed or item title
- Entry URIs are unique to make sure RSS readers don't treat episodes as duplicates
- Titles are unique to make sure RSS readers don't treat streams and downloads as duplicates
The bridge has been expanded to better cover the whole GQ magazine.
It should support all countries (provided they all use the same absurdly shitty publication system).
It is guaranteed to be only tested with sexactu articles (that I now obtain by loading Maïa Mazaurette author page).
This commit adds a new optional parameter 'limit' which can be used
to limit the number of items returned by this bridge (i.e. '&limit=10')
As requested in #669
* [Exceptions] Don't return header for bridge exceptions
* [Exceptions] Add link to list in exception message
This is an alternative when the button is not rendered
for some reason.
* [index] Don't return bridge exception for formats
* [index] Return feed item for bridge exceptions
* [BridgeAbstract] Rename 'getCacheTime' to 'getModifiedTime'
* [BridgeAbstract] Move caching to index.php to separate concerns
index.php needs more control over caching behavior in order to cache
exceptions. This cannot be done in a bridge, as the bridge might be
broken, thus preventing caching from working.
This also (and more importantly) separates concerns. The bridge should
not even care if caching is involved or not. Its purpose is to collect
and provide data.
Response times should be faster, as more complex bridge functions like
'setDatas' (evaluates all input parameters to predict the current
context) and 'collectData' (collects data from sites) can be skipped
entirely.
Notice: In its current form, index.php takes care of caching. This
could, however, be moved into a separate class (i.e. CacheAbstract)
in order to make implementation details cache specific.
* [index] Add '_error_time' parameter to $item['uri']
This ensures that error messages are recognized by feed readers as
new errors after 24 hours. During that time the same item is returned
no matter how often the cache is cleared.
References https://github.com/RSS-Bridge/rss-bridge/issues/814#issuecomment-420876162
* [index] Include '_error_time' in the title for errors
This prevents feed readers from "updating" feeds based on the title
* [index] Handle "HTTP_IF_MODIFIED_SINCE" client requests
Implementation is based on `BridgeAbstract::dieIfNotModified()`,
introduced in 422c125d8e and
simplified based on https://stackoverflow.com/a/10847262
Basically, before returning cached data we check if the client send
the "HTTP_IF_MODIFIED_SINCE" header. If the modification time is
more recent or equal to the cache time, we reply with "HTTP/1.1 304
Not Modified" (same as before). Otherwise send the cached data.
* [index] Don't encode exception message with `htmlspecialchars`
* [Exceptions] Include error message in exception
* [index] Show different error message for error code 0
The URI "https://facebook.com/username?_fb_noscript=1" returns two
posts per user. Some profiles, however, are very active, causing the
bridge to miss items if more than two posts are send within the cache
duration (5 minutes).
The alternative suggested in #669 is to use a different URI:
"https://facebook.com/pg/username/posts?_fb_noscript=1"
While the contents of this URI essentially look the same when viewed
in a browser, it actually returns more than 10 posts depending on the
profile.
References #669
Adds a new class 'ParameterValidator' to replace the functions from
'validator.php', separating private functions from 'validateData' to
class private functions in the process.
Instead of echoing error messages, adds messages to a private variable,
accessible via 'getInvalidParameters'.
BridgeAbstract now adds invalid parameter names to the error message.
- Use 'array_diff_key' instead of 'unset'
- Remove parameters for caches
By removing certain parameters for caches, the loading times can be
improved considerably:
* action: It doesn't matter which action the user took to generate
feed items.
* format: This has the biggest impact on performance, because cached
items are now shared between different formats (i.e. try switching
between Atom, Html and Mrss and compare previous vs. now). If a
server handles lots of requests, this may even reduce bandwidth if
the same contents are requested for different formats.
* _noproxy: The proxy behavior has no impact on the produced items,
so it can be ignored.
* _cache_timeout: This is another option which might impact performance
for some servers, especially if 'custom_timeout' has been enabled in
the configuration. Requests with different cache timeouts no longer
result in separate cache files.
* Fix typo with status field
* Comply with other bridges
Change the uri element of an item to point, not on the magnet link, but on the page, as similar bridges do.
* Improved to return name & uri matching with query
This change makes it possible for the feed reader to discover a title and url consistent with the user's search.
This error is caused by setting label::before { content: " "; },
which makes the first letter a whitespace on all labels, neccessary
for browsers that doesn't support the grid layout.
This commit clears the content if the browser supports the grid layout,
properly capitalizing labels again. If a browser doesn't support grid
layout, labels stay as they are provided by the bridge.
* Group common selectors
* Fix indentation using tabs
* Use same styles for number and text inputs
* Use grid layout for parameters
Introduces the grid layout for bridge parameters. All parameters are
arranged in a grid to improve readability. Read more on grid layouts
at
- https://www.w3schools.com/css/css_grid.asp
- https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Grid_Layout
Notice:
Grid layouts are not supported in very old browser versions:
https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Grid_Layout/CSS_Grid_and_Progressive_Enhancement
This is why @supports checks for browser support (not supported in IE)
https://developer.mozilla.org/en-US/docs/Web/CSS/@supports#Browser_compatibility
In case grid layout is not supported, the displayed form is usable
but not very pretty due to <br> being removed by this commit for
cosmetic reasons (breaks grid layout).
Unfortunately it doesn't seem possible to insert line breaks manually
via '::after { content: '\A' }' in cases where grid layout isn't
supported.
* Add padding to card parameters
Adds padding to parameters to improve readability. For bridges without
parameters (count($parameters) === 0), the parameter 'div' is no longer
created.
* Add colon ':' after label via CSS
* Capitalize first letter of label for readability
* Fix checkbox isn't aligned left
Sets the size of the checkbox to 20x20 px for good measure.
* Harmonize formatting
* Add new style to number and select boxes
References #797
* Add fallback solution for browsers without grid support
XML attributes need to have certain characters escaped to be valid. The title attribute can have double quotes in it which need to be properly encoded for attributes.
* [ZoneTelechargementBridge] Added Bridge for ww2.zone-telechargement1.org
Goal for this bridge is to follow the episode publication of a TV show
season while it's broadcasted on the TV.
* [AutoJMBridge] New Bridge
This bridge will show all the car offers AutoJM has for the model you
choosed and using your filter. Very useful to wait for a cheap price for
a new car !
This commit fixes#782 by updating the parameter value of 'Maison &
Jardin', but this means the user has to update his RSS Feed URL (.because
of the bridge structure, it would be a nightmare to fix it in another
way)
This commits add all the categories available on Dealabs Website.
* [ExtremeDownloadBridge] Added Bridge for ww1.extreme-d0wn.com
Goal for this bridge is to follow the episode publication of a TV show season
while it's broadcasted on the TV.
* Adding a bridge for theyetee.com
The bridge fetches daily shirts at theyetee.com.
The Yetee offers two new shirts each day, but you can buy them only for a few hours !
Unfortunately, the site don't provide RSS feed, so the only way to keep up to date on new shirt is their daily mailing ... until now !
* Debug mode improvements
- Improve debug warning message
- Restore error reporting in debug mode
- Fix 'notice' messages for unset fields
* Add parsing utility functions
html.php
- extractFromDelimiters
- stripWithDelimiters
- stripRecursiveHTMLSection
- markdownToHtml (partial)
bridges
- remove now-duplicate functions
- call functions from html.php instead
* [Anidex] New bridge
Anime torrent tracker
* [Anime-Ultime] Restore thumbnail
* [CNET] Recreate bridge
Full rewrite as the previous one was broken
* [Dilbert] Minor URI fix
Use new self::URI property
* [EstCeQuonMetEnProd] Fix content extraction
Bridge was broken
* [Facebook] Fix "SpSonsSoriSsés" label
... which was taking space in item title
* [Futura-Sciences] Use HTTPS, More cleanup
Use HTTPS as FS now offer HTTPS
Clean additional useless HTML elements
* [GBATemp] Multiple fixes
- Fix categories: missing "break" statements
- Restore thumbnail as enclosure
- Fix date extraction
- Fix user blog post extraction
- Use getSimpleHTMLDOMCached
* [JapanExpo] Fix bridge, HTTPS, thumbnails
- Fix getSimpleHTMLDOMCached call
- Upgrade to HTTPS as JE now offers HTTPS
- Restore thumbnails as enclosures
* [LeMondeInformatique] Fix bridge, HTTPS
- Upgrade to HTTPS as LMI now offers HTTPS
- Restore thumbnails using small images
- Fix content extraction
- Fix text encoding issue
* [Nextgov] Fix content extraction
- Restore thumbnail and use small image
- Field extraction fixes
* [NextInpact] Add categories and filtering by type
- Offer all RSS feeds
- Allow filtering by article type
- Implement extraction for brief articles
- Remove article limit, many brief articles are publied all at once
* [NyaaTorrents] New bridge
Anime torrent tracker
* [Releases3DS] Cache content, restore thumbnail
- Use getSimpleHTMLDOMCached
- Restore thumbnail as enclosure
* [TheHackerNews] Fix bridge
- Fix content extraction including article body
- Restore thumbnail as enclosure
* [WeLiveSecurity] HTTPS, Fix content extraction
- Upgrade to HTTPS as WLS now offers HTTPS
- Fix content extraction including article body
* [WordPress] Reduce timeout, more content selectors
- Reduce timeout to use default one (1h)
- Add new content selector (articleBody)
- Find thumbnail and set as enclosure
- Fix <script> cleanup
* [YGGTorrent] Increase limit, use cache
- Increase item limit as uploads are very frequent
- Use getSimpleHTMLDOMCached
* [ZDNet] Rewrite with FeedExpander
- Upgrade to HTTPS as ZD now offers HTTPS
- Use FeedExpander for secondary fields
- Fix content extraction for article body
* [Main] Handle MIME type for enclosures
Many feed readers will ignore enclosures (e.g. thumbnails) with no MIME type. This commit adds automatic MIME type detection based on file extension (which may be inaccurate but is the only way without fetching the content).
One can force enclosure type using #.ext anchor (hacky, needs improving)
* [FeedExpander] Improve field extraction
- Add support for passing enclosures
- Improve author and uri extraction
- Fix 'notice' PHP error messages
* [Pull] Coding style fixes for #802
* [Pull] Implementing changes for #802
- Fix coding style issues with str append
- Remove useless CACHE_TIMEOUT
- Use count() instead of $limit
- Use defaultLinkTo() + handle strings
- Use http_build_query()
- Fix missing </em>
- Remove error_reporting(0)
- warning CSS (@LogMANOriginal)
- Fix typo in FeedExpander comment
* [Main] More documentation for markdownToHtml
See #802 for more details
The bridge would generate empty titles if the content is longer than
50 characters, but doesn't have further spaces in it. With this commit
the title is correctly generated based on the contents, taking missing
spaces into account.
References #786
https://cad-comic.com/ now provides feeds at
- https://cad-comic.com/feed (rss)
- https://cad-comic.com/feed/atom (atom)
Thus multiple alternatives are available to choose from, making this
bridge obsolete:
- FilterBridge (using one of the feeds above)
- WordPressBridge (on the main site)
- One of the two available feeds
References #752
This commit fixes an issue caused by self closing tags not supported
by simplehtmldom (<source>).
Adds a monkey patch to extend simplehtmldom with the ability to detect
that particular tag. Most of the code added is copied directly from
simplehtmldom (see vendor/simplehtmldom) with adjustments to account
for RSS-Bridge formatting.
Related to: https://sourceforge.net/p/simplehtmldom/bugs/83/
Notice: The tag itself is valid according to Mozilla:
The HTML <picture> element serves as a container for zero or more
<source> elements and one <img> element to provide versions of an
image for different display device scenarios. The browser will
consider each of the child <source> elements and select one
corresponding to the best match found; if no matches are found
among the <source> elements, the file specified by the <img>
element's src attribute is selected. The selected image is then
presented in the space occupied by the <img> element.
-- https://developer.mozilla.org/en-US/docs/Web/HTML/Element/picture
References #753
Adds a new action '?action=list' to return a list of bridges as JSON formatted text. Each bridge brings following information:
- status (active/inactive)
- uri
- name
- parameters
- maintainer
- description
For inactive bridges only the status is returned.
Bridges that cannot be instantiated are considered inactive.
Adds unit test for bridge implementations:
- Custom functions must be in protected or private scope
- getName() must return a valid string (non-empty)
- getURI() must return a valid URI
- Each bridge must define constants for NAME, URI, DESCRIPTION and
MAINTAINER. CACHE_TIMEOUT and PARAMETERS are optional.
The unit test is written for PHPUnit 6.x and will automatically be
tested by Travis-CI for PHP 7.0 (see .travis.yml).
Remarks:
Unit tests for bridge data were scrapped in #378 for complexity
reasons (tests would have to be maintained for each bridge). This
unit test, however, is written for testing all bridges without
taking specific implementation details into account.
* use defaultLinkTo
* remove duplicate video links
* remove line ending before "Reposted" label
* return newline before reposted string
* remove comments
* use video links that won't require login
* set title if video has no title
Adds a new option '&title_from_content=on' to build the title for feed
items from the feeds content. The title is generated from the first
whitespace after 50 characters of the content or the entire content if
the total size is lower than 50 characters.
References #587
This commit fixes a few things related to static::URI
1) Remove trailing slash from the URI to simplify using 'defaultLinkTo'
2) Use static::URI instead of self::URI for consistency
3) Remove custom implementation of 'defaultLinkTo'
Images are collected for each post and added to enclosures. Images or
animtions from lh3.googleusercontent.com are specifically handled in
order to return the animated version of the gif and the original sized
image (this is normally taken care of by JS in the browser).
Adds a new bridge for https://gist.github.com
The bridge generates feeds for comments on a particular gist based on
the gist ID or full URI. For better readability the general behavior
of code sections is manually restored with the original CSS styles
from GitHub.
New bridge for Skimfeed: https://skimfeed.com
Generates feeds for all features of Skimfeed:
- News (the ones displayed on the front page)
- Hot topics ("What's Hot" section on the front page)
- Tech news (preconfigured feeds in the menu bar)
- Custom feeds (using the configuration system of Skimfeed), see
https://skimfeed.com/custom.php
The number of items returned by the bridge can be limited for all
categories ('&limit=...'). This parameter is optional, all categories
are unlimited by default!
Authors are added with HTML anchors in order to allow quick navigation
to source channels.
The bridge ships with developer tools to auto-generate lists in the
future (especially useful for 'Tech news'!)
References #748
Following changes in the JSON data and selecting images for the
content (320x240 or bigger) and enclosure (largest version). All of
the data is now extracted from the JSON data instead of parsing the
DOM.
References #754
[html] Generate index and bridge cards using separate clases
Move HTML generating code from 'index.php' to 'Index.php', separating components into static functions.
Move HTML generation code for bridge cards from 'html.php' to 'BridgeCard.php', separating components into static functions.
Removing this bridge for two reasons:
1) The service moved from www.cpasbien.cm to www.torrents9.blue,
changing the layout in the process (incompatible).
2) The new site is permanently protected by Cloudflare IUAM, making
it inaccessible by RSS-Bridge.
While it would certainly be possible to rewrite the bridge to work
with the new layout, the site is still inaccessible.
References #605
Adds detection for servers responding with Cloudflare challenges,
throwing a server error if detected:
"The server responded with a Cloudflare challenge, which is not
supported by RSS-Bridge! If this error persists longer than a week,
please consider opening an issue on GitHub!"
This is supposed to support maintainers to identify broken bridges
for sites with Cloudflare enabled permanently. It doesn't circumvent
the protection in any form or shape!
The Cloudflare challenge is detected by analyzing the last response
header received from the server. If the HTTP Code is not 200 (OK)
and the server name contains 'cloudflare' ('Server: cloudflare'),
RSS-Bridge assumes the server responded with a challenge.
The header parsing is based on https://stackoverflow.com/a/18682872
This commit adds a new bridge for http://www.instructables.com. This bridge
currently supports fetching content by category (all categories available 200+),
using available filters (featured, recent, popular, views, contest winners).
Adds duration limits (minimum duration, maximum duration) for all
modes (user/id/playlist/search). Duration limits are optional, so
existing subscriptions don't break.
The limits are specified by two separate parameters, each of which
is optional:
- `&duration_min=` (minimum duration in minutes, default: -1)
- `&duration_max=` (maximum duration in minutes, default: INF)
If duration limits are specified in either user, id or playlist mode,
the bridge defaults to fetching data from HTML intead of XML feeds,
which requires more bandwidth and takes longer, because each video is
loaded individually!
References #670
Adds matching for URIs to the search bar, using the format
<scheme>://<host>/<path>
Searching by URI scheme is also supported:
"http://" (returns all bridges with 'http' scheme)
"https://" (returns all bridges with 'https' scheme)
The following examples are equivalent and will return both of the
Facebook bridges (FacebookBridge and FB2Bridge):
"https://www.facebook.com/facebook"
"https://www.facebook.com/facebook?..."
"https://www.facebook.com"
"http://www.facebook.com"
"https://facebook.com"
"http://facebook.com"
"facebook.com"
"facebook"
Notice: When the URI scheme is omitted, the search algorithm falls back
to regex matching. Searching for "www.facebook.com" doesn't work, as it
is missing the schema and doesn't match via regex!
Omitting the 'www.', however, does work. This was a design decision for
some bridges specify their URI with and others without 'www.'
A search term can still be specified in the browser URL using parameter
'q' => '?q=searchterm'.
References #743
The previous context is now labeled 'User', while the new context is
labeled 'Group'. The existing code was not changed, instead new group*
functions were implemented to handle groups.
The general principle of capturing groups is the same as done for users
with adjustments to account for different HTML structures.
Captcha responses are currently not supported for groups! There doesn't
seem to be a way to trigger them consistently, which makes it hard to
handle them properly.
Features of the group context:
- The feed title is based on the group name
- The group URI used for capturing is returned for the feed URI
- Author names and timestamps are reproduced from the source
- Post titles are reproduced from the source if they exist, otherwise
the title is build manually from the author name and the content
- Original contents are included with the feed
- All images are attached as enclosures as well
Closes #
Supports latest news and profiling a given ETF in Englisch, German
or Italian language. Cover images are attached as enclosures and not
as part of the content.
News:
Optionally loads the full article for each news item. Some articles
may include scripts to provide interactive graphs. These scripts are
removed as they would be rendered as pure text and a message is shown
instead: "[Content removed! Visit site to see full contents!]"
Profile:
Optionally includes the ETF strategy and description.
* Move configuration in its own class in order to reduce the verbosity of index.php
* Add authentication mechanism using HTTP auth
* Add a method to get the config parameters
* Remove the installation checks from the index page
* Log all failed authentication attempts
isoHunt has discontinued services due to legal reasons and is now
accessible via https://isohunts.to
While it is certainly possible to rewrite the bridge to fetch some
information from the new site, it wouldn't be able to provide as
much functionality as before. This is due to isoHunt having removed
all searching and filtering options, only providing static HTML pages
for general categories (anime, movies, etc...). Those pages, however,
are heavily broken.
Unless someone is interested in monitoring the general categories
the effort of upgrading the bridge to the new site is not worth taking
time for.
Users of isoHunt are asked to make use of their client application,
as they don't provide online services anymore (it's now in the darknet)
Here is the statement from isoHunt:
"Due to hard regulations and security issues for bittorrent users, we
have moved into a more secure and even faster district of the internet!
[...]
Torrent Downloads have a high risk of getting legal problems. That is
why we do not offer torrentfiles any more. [...]"
-- source: https://isohunts.to
Allows users to paste facebook links as user name. The link must contain
the correct host (www.facebook.com) and a valid path (/user-name/...).
The first part of the path is used for the user name. Errors are returned
in case something went wrong.
References #706
Reviews are provided the same way as summary posts and therefore returned
as separate feed item for each review. This commit adds a new option
'&skip_reviews=on' to skip reviews entirely.
References #706
The media_type parameter was recently replaced by media_type_u (for
user mode) and media_type_h (for hashtag mode). This was necessary
in order to add the media type 'story' only for the user mode.
"The reason for that is that RSS-Bridge supports multiple parameters
with the same name if and only if they contain the exact same value.
Here, hashtags don't have stories, so it would not be possible to
pass "story" as a parameter. This is a design mistake that I made
when I added support for hashtags."
-- 8770c87389 (r28871502)
However as pointed out this change breaks existing feeds as the
parameter name is no longer compatible to previous implementations.
This commit changes the implementation to provide the old media_type
parameter globally and check for invalid options on each request. If
a user uses the 'story' option in history mode the bridge returns a
client error.
references 8770c87
references #694fixes#696fixes#699fixes#701
Uses the parse_ini_file function to load default settings from the default configuration file 'config.default.ini.php'. Optionally loads custom settings from 'config.ini.php' to replace the default
values.
* [DealabsBridge] Follow site changes, fix unhandled case
- Fixed the case where no discount was shown
- Changed some CSS class to follow the website changes
Now, entry title is optionnal and may be found in h3 HTML element.
Entry content is mandatory and may be found in div[class="item-content"] HTML element.
Moreover, the title may contain simple quotes (here, encoded) so the bridge have to decode first to apply format library function. In case we don't do that, the format function double encode the quote and something like &#039; could appear.
* [VkBridge] Convert special HTML entities to characters in pageName
* [VkBridge] Generate feed item title
* [VkBridge] Remove double backslashes in feed item link
* [VkBridge] Unpin post if pinned
* [VkBridge] Mark reposted messages
* [VkBridge] Correct external link parsing
* [VkBridge] Added article parsing
* [VkBridge] Added video parsing
* [VkBridge] Added photo parsing
* [VkBridge] Added album link parsing
* [VkBridge] Added one more external link selector
* [VkBridge] Using array of link selectors to remove
* [VkBridge] Added document parsing
* [VkBridge] Added sign parsing
* [VkBridge] Fixed incorrect sorting with pinned item
* [VkBridge] More methods to parse documents
* [VkBridge] Save fallback if page name element not found
* [VkBridge] Using post signed as feed item author
* [VkBridge] Fixed document link
* [VkBridge] Coding policy fixes
NotAlways right found it necessary to remove their RSS feeds recently. This is a *simple* bridge to grab the ones on the front page. It allows you to filter the articles based on their classification (right, working, romantic, related, learning, friendly, hopeless, unfiltered, or all).
The limit was used to specify the number of pages to return from a given
topic which resulted in the number of returned items variing between one
and however many entries are listed on one page.
This commit changes the implementation for the limit to keep loading more
pages until the specified limit is reached. Excessive elements are removed
in order to return the exact amount of items specified by the limit.
This behavior is closer to how other bridges are implemented and makes it
more natural to use without being too confusing. Existing queries must be
updated to account for the new limit.
References #657
cURL is a powerful library specifically designed to connect to many
different types of servers with different types of protocols. For
more detailed information refer to the PHP cURL manual:
- http://php.net/manual/en/book.curl.php
Due to this change some parameters for the getContents function were
necessary (also applies to getSimpleHTMLDOM and getSimpleHTMLDOMCached):
> $use_include_path removed
This parameter has never been used and doesn't even make sense in
this context; If set to true file_get_contents would also search
for files in the include_path (specified in php.ini).
> $context replaced by $header and $opts
The $context parameter allowed for customization of the request in
order to change how file_get_contents would acquire the data (i.e.
using POST instead of GET, sending custom header, etc...)
cURL also provides facilities to specify custom headers and change
how it communicates to severs. cURL, however, is much more advanced.
- $header is an optional parameter (empty by default). It receives
an array of strings to send in the HTTP request header.
See 'CURLOPT_HTTPHEADER':
"An array of HTTP header fields to set, in the format
array('Content-type: text/plain', 'Content-length: 100')"
- php.net/manual/en/function.curl-setopt.php
- $opts is an optional parameter (empty by default). It receives
an array of options, where each option is a key-value-pair of
a cURL option (CURLOPT_*) and it's associated parameter. This
parameter accepts any of the CURLOPT_* settings.
Example (sending POST instead of GET):
$opts = array(
CURLOPT_POST => 1,
CURLOPT_POSTFIELDS => '&action=none'
);
$html = getContents($url, array(), $opts);
Refer to the cURL setopt manual for more information:
- php.net/manual/en/function.curl-setopt.php
> $offset and $maxlen removed
These options were supported by file_get_contents, but there doesn't
seem to be an equivalent in cURL. Since no caller uses them they are
safe to remove.
Compressed data / Encoding
By using cURL instead of file_get_contents RSS-Bridge no longer has
to handle compressed data manually.
See 'CURLOPT_ENCODING':
"[...] Supported encodings are "identity", "deflate", and "gzip".
If an empty string, "", is set, a header containing all supported
encoding types is sent."
- http://php.net/manual/en/function.curl-setopt.php
Notice: By default all encoding types are accepted (""). This can
be changed by setting a custom option via $opts.
Example:
$opts = array(CURLOPT_ENCODING => 'gzip');
$html = getContents($url, array(), $opts);
Proxy
The proxy implementation should still work, but there doesn't seem
to be an equivalent for 'request_fulluri = true'. To my understanding
this isn't an issue because cURL knows how to handle proxy communication.
Requesting a username with a leading slash would cause error 500
because the requested URI would contain two slashes in a row.
For example username "/test" would result in:
https://facebook.com//test
References #628
* [BridgeAbstract] Implement customizable cache timeout
The customizable cache timeout is used instead of the default cache
timeout (CACHE_TIMEOUT) if specified by the caller.
* [index] Add new global parameter '_cache_timeout'
The _cache_timeout parameter is an optional parameter that can be
used to specify a custom cache timeout. This option is enabled by
default.
It can be disabled using the named constant 'CUSTOM_CACHE_TIMEOUT'
which supports two states:
> true: Enabled (default)
> false: Disabled
* [BridgeAbstract] Change scope of 'getCacheTimeout' to public
* [html] Add cache timeout parameter to all bridges
The timeout parameter only shows if CUSTOM_CACHE_TIMEOUT has been set
to true. The default value is automatically set to the value specified
in the bridge.
* [index] Disable custom cache timeout by default
The mbstring extension is required by all formats in order to convert multi-
byte characters to UTF-8. This commit adds an extension check to throw an
error message if the extension is not enabled.
The author name is parsed by searching a string within the entire
HTML document:
$author = $html->innertext;
$author = substr($author, strpos($author, '"author=') + 8);
$author = substr($author, 0, strpos($author, '\u0026'));
This solution will return big portions of the HTML document if
the strpos function returns zero (not found).
This commit replaces the previous implementation by searching for
a specific script tag and making use of the JSON data inside it.
References #580
This bridge returns feeds for any URI that is compatible with the
IPB implementation (currently 4.x). Older versions might work, but
there is no guarantee.
Only forum and topic URIs are supported!
The bridge automatically checks if natural feeds are available (by
adding '.xml' to the URI). If so the feed is returned. Otherwise
the bridge will attempt to identify the content type and build a
feed accordingly.
Valid URIs are forums and topics. For forums the first page is
returned, for topics the last one. Elements are ordered such that
the latest entry is returned first (oldest-to-newest)
The optional parameter '&limit=' specifies how many pages should
be loaded (default: 1). Topics are loaded in reverse order.
=> Does not work with forums!
Images are provided as enclosures and scaled to a max-size of
400x400 pixels by default (Except for natural feeds).
The content is filtered before being returned:
- Unnecessary tags are removed (iframes, etc...)
- Styles for blockquotes are restored (grey background)
Closes#507
Travis CI upgraded the linux build environment from Ubuntu Precise
to Ubuntu Trusty with Trusty becoming the default build environment
as of August 2017:
https://docs.travis-ci.com/user/reference/overview/
A bug in the configuration of the Ubuntu Trusty distro causes all
builds except nightly to fail. The PHP include_path is set to
include_path='.:/home/travis/.phpenv/versions/5.6.31/share/pear'
instead of
include_path='.:/home/travis/.phpenv/versions/5.6.31/lib/php/pear'
which causes phpcs to fail because it cannot resolve import paths.
This commit adds a hotfix to .travis.yml that circumvents the
issue by overwriting the include_path during initialization. This
hotfix should be removed once a solution is found.
This bug is tracked via
https://github.com/travis-ci/travis-ci/issues/8487
Travis CI recently updated the default distribution from Ubuntu
Precise to Ubuntu Trusty, which causes all builds except nightly
to fail.
For unknown reasons phpcs is unable to locate "PHP/CodeSniffer/
autoload.php" causing it to fail with a fatal error
The root cause of the failure is unknown. We explicitly return to
the previous build system (Ubuntu Precise) for builds to work again.
See the migration guide for reference:
https://docs.travis-ci.com/user/precise-to-trusty-migration-guide/
Notice: Ubuntu Precise is retired as of September 2017 and will be
decommissioned in the near future:
https://docs.travis-ci.com/user/reference/overview/
- If no there are no results, an explicit message is now returned
- Commas are now following the coding policy
- Lines are no longer more than 80 chars when possible
Added additional instructions for: 'username search' and 'category
search' next to instructions for 'keyword search'.
Changed variable name from underscore to camelCase.
Videos that are part of a playlist have the playlist ID encoded in
the URI. When loading the video info the page contents change unex-
pectedly due to the playlist being part of the page.
This removes any trailing parameters from the video ID in order to
ensure only pure videos are loaded at all times.
Providing no value for an optional numeric parameter results in
error "Parameter *** is invalid!"
This is caused by the validation function ignoring the 'required'
attribute when loading and checking input parameters.
This commit adds checks to determine whether the 'required' attri-
bute is defined and active before returning the error message.
References #570:
Loading the page with an URL fragement (#bridge-*) should result in
the bridge showing all parameters by default. Unfortunately this is
not possible using PHP, which is why a new JavaScript function is
needed (select.js)
That way, when returning from a bridge ('back to rss-bridge') will
keep the selected bridge active (only works for HTML format).
f757d7d1a5 introduced a bug where all
text input boxes were centered instead of just the search bar.
In order for this to work properly the global styles must be applied
before specific styles for the search bar.
The search bar doesn't feel right if the placeholder is centered,
while the text cursor is left-aligned. The cursor should appear
instead of the placeholder (at the same position).
Added styles to center the text cursor and hide the placeholder
when selecting the input field.
Tested in:
- Firefox 54 & 55
- Chromium 60 (compatible with Chrome 60)
- Microsoft Edge (partially working!)
--- Microsoft Edge ---
Due to a bug in the Microsoft Edge browser, the text cursor is not
centered as long as the placeholder is defined (which it is always)
More information:
https://stackoverflow.com/a/33224868
Official bug report:
https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/4468563/
----------------------
Whitespace at the beginning of feeds causes parsing errors. This is
an example using an ill-formatted RSS feed:
"XML or text declaration not at start of entity"
-- https://validator.w3.org
This commit automatically removes all proceeding and trailing white-
space from the source content before resume parsing.
On Oct 27, 2016 the discontinuation of Vine was announced:
https://medium.com/@vine/important-news-about-vine-909c5f4ae7a7
"Today, we are sharing the news that in the coming months we’ll be
discontinuing the mobile app."
https://vine.co/ is still online, but has been put into an archive
indefinitely. As the site does not allow further uploads, this
bridge serves no further purpose.
Matching whitelisted bridges using a case-insensitive match makes
sense for following reasons:
- Wrong upper/lower case spelling in the whitelist is not easily
discovered. Example: Misspelling 'Youtube' as 'YouTube' will not
show the 'Youtube' bridge (while it is expected to show)
- Two bridges with the same name but different letter casing are
discouraged to prevent confusion and keep the project compatible
with Windows machines
Tags are embedded in the 'title' attribute instead of 'alt' as
defined by the ancestor (DanbooruBridge).
The 'title' attribute also contains statistics data ('score:...',
'rating:...') that is now filtered by a custom implementation of
the 'getTags' function (elements that contain a colon are removed)
Closes#560
* The cache folder requires write permissions at all times
* The whitelist file requires write permissions if it does not
exist (can be created manually)
All formats except HTML return & instead of & in URLs causing
all links with parameters (...&id=...) to break.
Facebook does not return valid HTML URIs but instead provides them
with all special characters encoded (like using htmlspecialchars).
This seems to be related to the page being build almost entirely of
script blocks.
This commit adds htmlspecialchars_decode() to URI and content to
reverse the encoding.
References #550
- Do not add spaces after opening or before closing parenthesis
// Wrong
if( !is_null($var) ) {
...
}
// Right
if(!is_null($var)) {
...
}
- Add space after closing parenthesis
// Wrong
if(true){
...
}
// Right
if(true) {
...
}
- Add body into new line
- Close body in new line
// Wrong
if(true) { ... }
// Right
if(true) {
...
}
Notice: Spaces after keywords are not detected:
// Wrong (not detected)
// -> space after 'if' and missing space after 'else'
if (true) {
...
} else{
...
}
// Right
if(true) {
...
} else {
...
}
Add additional information to error message:
- Name of the bridge
- Possible solutions
- Error description
- Error code
- Error message
* Output type changed from 'text' to 'html'
* Added styles for the error page
* Added a button to remotely open a GitHub issue
Closes#525
This replaces the 'novideo' parameter with 'media_type' in order
to filter for specific content types. Currently supported:
- 'all': Returns all posts (default)
- 'video': Returns only posts including videos
- 'novideo': Returns only posts that don't include videos
References #553
This adds a new option 'novideo' that can be set to 'on' or 'off'
in order to skip posts that include facebook videos (does not work
for linked videos like YouTube). This option is 'off' by default.
References #533
Adds a new option 'media_type' to select from three choices:
- 'all' (Both): Returns pictures and videos (default choice)
- 'picture': Returns only pictures
- 'video': Returns only videos
References #553
This fixes broken URIs in the output data caused by duplicate domain
names caused by sites (descendant class Delbooru) providing absolute
URIs instead of relative ones.
References #552
The title attribute includes tags (anchors) instead of raw text.
While this works fine in a browser, using a raw format like plain-
text or json breaks with expected behavior.
This commit changes the order in which functions are applied. By re-
moving anchors AFTER fixing the title, the final result does not
include tags and the title is still fixed.
This bug was introduced by d81b61ccfa
References: #546
Twitter now requires login to access "Tweets & Replies" which
breaks feeds using the default behavior. Using the "Without
replies" option still works.
This commit makes the "Without replies" option default. That way
existing feeds will return contents again. The parameter can still
be checked but its status has no effect anymore.
Notice: The parameter should not be removed as that would cause
any feed using the parameter to stop working because of "Invalid
parameter"
References #544
This commit improves readability of tweets by adding spaces before
anchors in the text.
- Hide "invisible" tags which were not rendered hidden because of
missing CSS.
- Fix spacing between anchors
Reported via #539
If no accepted languages are specified Facebook will guess your
language. This guess can go horribly wrong if your server does not
provide origin information.
This adds a context header with language information when retrieving
page contents. The accepted languages are read from the list of
accepted languages specified by the web browser of the requester.
References #530
This improves the translation of regular text to anchors by adding
support for additional characters '?&=-_' to fix common URLs.
Notice: The regex pattern is by no means complete. That means it is
likely to break in the future. More sophistiated solutions however
are insanely complex. See: http://stackoverflow.com/a/190405
References #520
Twitter provides an easy way to receive various image sizes based
on the same image URI:
https://dev.twitter.com/overview/api/entities-in-twitter-objects
We support different sizes: thumb, small, medium and large. The
media_url defaults to medium but you can retrieve the media in
different sizes by appending a colon + the size key (for example:
https://pbs.twimg.com/media/A7EiDWcCYAAZT1D.jpg:thumb).
-- Twitter Developer Documentation
TwitterBridge now makes use of this feature in order to provide
thumbnail images in the content and original sized images as
enclosures.
References #526
Previously feed contents were rendered as one block of text with
no structure. This brings back the structure of original video
descriptions and makes links in the description work again.
References #520
Re-requesting the same feed in normal mode (not debug mode)
returns the cached version. The name and URI of the feed
however was not returned.
This adds checks to getName() and getURI() in order to return
the cached infos when using the cached version.
Notice: For this to work correctly all bridges must call to
parent::getName() and parent::getURI() respectively if the
queriedContext is not defined. Example:
switch($this->queriedContext){
case 'My context':
// do your stuff here!
break;
default: parent::getName();
}
This adds new features to show quotes and pictures in feeds.
Quotes will show up on top of a tweet and are separated from
the quoting feed by a horizontal line.
Pictures that are embedded in the tweet will be captured and
attached to the feed using enclosures. By default the picture
will also be shown in the feed itself. This can be disabled
using the option '&noimg=on'
Some codes are now split into separate functions so they can be used
for tweets and quotes alike.
Changed behaviour of the checkbox to use the custom parser when
active. That way if the parameter is missing the default value
applies and the feed returns from the provided RSS
Reported via #498
Previously summary posts were ignored which resulted in the last
two posts not showing up in the feed (the latest two are shown in
the summary post).
Now summary posts are treated like regular posts, returning them
as part of the regular feed.
References #502, #505
Due to breaking DOM changes this bridge required re-implementation.
With this fix the brige will make use of the JSON data embedded in
the returned HTML. The content returned for all contexts is similar
with only a few differences due to limitations of the JSON.
Feeds returned for a given username and board will by default make
use of the provided RSS feed instead of using the custom filter.
This bahaviour can be changed by setting the optional parameter
'&r=off' (on by default)
Notice: The JSON data for userdata and search results is very
different, so two functions were implemented to account for that.
References #498
* const DOMAIN is not supported, it must be const URI
* strtotime should be used instead of date_parse in order to
receive a valid integer
* Some small readability enhancement
This bridge generates feeds for a given search term, optionally
adds the picture to the content and allows for additional query
extensions (GET parameters) to be passed to the bridge. That
way custom filter can be applied without the need to reproduce
them in this bridge (they got a lot!)
Etsy provides a good set of feeds as described here:
https://www.etsy.com/help/article/100
(so there is no need to include them here)
References #492
This bridge will fetch contents from https://wikileaks.org
Available options are:
- Category: Defines a list of categories to select from
- Show teaser: Defines whether to show the teaser or not
Notice: Feeds provided by WikiLeaks do not work, see
https://wikileaks.org/wiki/RSSCloses#489
User names can either be an ID (series of numbers), or an actual
name, where the name always starts with a '+'.
This commit adds a check for automatically fixing provided user
names which are missing the '+'.
- Do not force language via HTTP header
The header enforced the language to be french which caused problems parsing
the exact time due to spellings (strtotime cannot work with 'semaines'). If
further issues are experienced try forcing en-us instead.
=> This should really be done in the RSS-Bridge core
- Fix loading problems due to pinned articles
Pinned articles do not provide a timestamp. Building the timestamp step-by-step
solves parsing errors.
- Use class names instead of CSS paths
CSS paths change based on the article. Pinned articles provide a different
DOM structure which caused parsing errors.
Reported via #499
Adds a new bridge to fetch contents from https://usbeketrica.com/
Feeds are build from cards displayed on the front page
This bridge provides two options:
- limit: Defines how many articles are returned
- fullarticle: Defines whether or not the full article is retured
Requested via #457
This bridge was broken due to DOM changes. This commit fixes
most of the broken code. Hashtags do no longer work because
they are no longer supported/provided.
The timing might be off as the source only provides a rough
relative value like '1 hour' or '1 year'.
Closes#485
json_encode causes high memory footprint on large input data,
where serialize is less problematic.
Example: When using AcrimedBridge items contain pictures in
raw format (entire picture) which leads to a file size of about
2MB using serialize. json_encode will allocate about 98MB of
memory for encoding, causing memory exhausion errors (PHP
allows for 128MB of memory by default)
This bridge returns feeds for each section (via given separator)
from a given MoinMoin compatible wiki.
The separator can be any tag of the following:
- h1
- h2
- h3
- li
- a
The number of items returned can be specified.
For anchor tags (a) the bridge can optionally follow the anchor to
the linked page and return it as content.
When declaring a function
- Do not add a space before a comma
- Add a space after a comma
- Add a space after an equal sign
Example:
function myFunction($x, $y, $z = null){...}
When calling a function
- Do not add a space before the opening parenthesis
- Do not add a space after the opening parenthesis
- Do not add a space before the closing parenthesis
- Do not add a space before a comma
- Add a space after a comma
Example:
myFunction('x', 'y', 'z');
This bridge is a mashup of the existing FlickrExploreBridge by sebsauvage
and FlickrTagBridge by erwang. It provides the same functionality as one
single bridge.
Instead of utilizing API requests for each element, the information
is now read directly from the source page, which provides information
as JSON data embedded in a script block.
The author name is returned for each element.
Improves the title and optionally adds the description if available
This adds a bridge for bugzilla.kernel.org to provide feeds for
bug comments without the need of registering an email address.
This implementation makes use of the print preview feature that
reduces bandwidth by a small margin.
Provides options to specify the number of comments to return as
well as the sorting order (latest first or oldest first)
Maintainer should be set for all bridges. Using git blame to
determine who provided the most code to the files. This is
obviously not a good solution, feel free to insert own names
getURI and getName should fall back to parent::getURI or
parent::getName respectively if it cannot build propper
return values.
Functions defined by bridges should be made private to
prevent confusion with inherited functions
* add function getExtraInfos() to BridgeAbstract
* replace call to $bridge->getName() and $bridge->getURI() by $bridge->getExtraInfos()
replace call to $bridge->getName() and $bridge->getURI() by $bridge->getExtraInfos() defined by default in BridgeAbstract.
So we could pass additionals ExtraInfos from custom bridges to custom formats.
All formats now support multiple enclosures. RSS
will show a warning if more than one enclosure
is used since many feed reader don't support
multiple enclosures with RSS (also not clearly
specified in the specification)
All bridges failed due to missing 'enclosures' element in
the items array. With this commit all formats (ATOM, RSS
and HTML) provide support for a single 'enclosure' element
PHP operator '===' is the only strict way to mix up the value '0' and
the value 'FALSE'.
The function saveData of the FileCache tests if the write of the cache
files was done with success and raise an Exception if not. The test was
done without the '===' operator, and if the data is 0 bytes long the
error message says there is a permission error, which is false.
A data 0 bytes long is another issue, either in the json_encode function
either in the Bridge, but not a permission issue.
* [Add a new line at the end of a file](https://github.com/RSS-Bridge/rss-bridge/wiki/Whitespace#add-a-new-line-at-the-end-of-a-file)
* [Do not add a whitespace before a semicolon](https://github.com/RSS-Bridge/rss-bridge/wiki/Whitespace#add-a-new-line-at-the-end-of-a-file)
* [Do not add whitespace at start or end of a file or end of a line](https://github.com/RSS-Bridge/rss-bridge/wiki/Whitespace#do-not-add-whitespace-at-start-or-end-of-a-file-or-end-of-a-line)
* [Use tabs for indentation](https://github.com/RSS-Bridge/rss-bridge/wiki/Indentation#use-tabs-for-indentation)
* [Maximum line length](https://github.com/RSS-Bridge/rss-bridge/wiki/Maximum-line-length)
* [The maximum line length should not exceed 80 characters](https://github.com/RSS-Bridge/rss-bridge/wiki/Maximum-line-length#the-maximum-line-length-should-not-exceed-80-characters)
* [Parameters with default values must appear last in functions](https://github.com/RSS-Bridge/rss-bridge/wiki/Functions#parameters-with-default-values-must-appear-last-in-functions)
* [Do not add spaces after opening or before closing bracket](https://github.com/RSS-Bridge/rss-bridge/wiki/Functions#do-not-add-spaces-after-opening-or-before-closing-bracket)
* [Structures must always be formatted as multi-line blocks](https://github.com/RSS-Bridge/rss-bridge/wiki/Structures#structures-must-always-be-formatted-as-multi-line-blocks)
* [Use PascalCase for class names](https://github.com/RSS-Bridge/rss-bridge/wiki/Classes#use-pascalcase-for-class-names)
* [Do not use final statements inside final classes](https://github.com/RSS-Bridge/rss-bridge/wiki/Classes#do-not-use-final-statements-inside-final-classes)
* [Do not override methods to call their parent](https://github.com/RSS-Bridge/rss-bridge/wiki/Classes#do-not-override-methods-to-call-their-parent)
* [abstract and final declarations MUST precede the visibility declaration](https://github.com/RSS-Bridge/rss-bridge/wiki/Classes#abstract-and-final-declarations-must-precede-the-visibility-declaration)
* [static declaration MUST come after the visibility declaration](https://github.com/RSS-Bridge/rss-bridge/wiki/Classes#static-declaration-must-come-after-the-visibility-declaration)
about: Use this template for requesting a new bridge
title: Bridge request for ...
labels: Bridge-Request
assignees: ''
---
# Bridge request
<!--
This is a bridge request. Start by adding a descriptive title (i.e. `Bridge request for GitHub`). Use the "Preview" button to see a preview of your request. Make sure your request is complete before submitting!
Notice: This comment is only visible to you while you work on your request. Please do not remove any of the lines in the template (you may add your own outside the "<!--" and "- ->" lines!)
-->
## General information
<!--
Please describe what you expect from the bridge. Whenever possible provide sample links and screenshots (you can just paste them here) to express your expectations and help others understand your request. If possible, mark relevant areas in your screenshot. Use the following questions for reference:
-->
- _Host URI for the bridge_ (i.e. `https://github.com`):
- Which information would you like to see?
- How should the information be displayed/formatted?
- Which of the following parameters do you expect?
- [X] Title
- [X] URI (link to the original article)
- [ ] Author
- [ ] Timestamp
- [X] Content (the content of the article)
- [ ] Enclosures (pictures, videos, etc...)
- [ ] Categories (categories, tags, etc...)
## Options
<!--Select options from the list below. Add your own option if one is missing:-->
- [ ] Limit number of returned items
- _Default limit_: 5
- [ ] Load full articles
- _Cache articles_ (articles are stored in a local cache on first request): yes
<!--Be aware that some options might not be available for your specific request due to technical limitations!-->
<!--
## Additional notes
Keep in mind that opening a request does not guarantee the bridge being implemented! That depends entirely on the interest and time of others to make the bridge for you.
You can also implement your own bridge (with support of the community if needed). Find more information in the [RSS-Bridge Wiki](https://github.com/RSS-Bridge/rss-bridge/wiki/For-developers) developer section.
rss-bridge is a PHP project capable of generating ATOM feeds for websites which don't have one.
RSS-Bridge is a PHP project capable of generating RSS and Atom feeds for websites that don't have one. It can be used on webservers or as a stand-alone application in CLI mode.
Supported sites/pages (main)
**Important**: RSS-Bridge is __not__ a feed reader or feed aggregator, but a tool to generate feeds that are consumed by feed readers and feed aggregators. Find a list of feed aggregators on [Wikipedia](https://en.wikipedia.org/wiki/Comparison_of_feed_aggregators).
Supported sites/pages (examples)
===
*`FlickrExplore` : [Latest interesting images](http://www.flickr.com/explore) from Flickr
*`GoogleSearch` : Most recent results from Google Search
*`GooglePlus`: Most recent posts of user timeline
*`Twitter`: Return keyword/hashtag search or user timeline
*`Identi.ca` : Identica user timeline (Should be compatible with other Pump.io instances)
*`YouTube` : YouTube user channel, playlist or search
*`Cryptome` : Returns the most recent documents from [Cryptome.org](http://cryptome.org/)
*`DansTonChat`: Most recent quotes from [danstonchat.com](http://danstonchat.com/)
*`DuckDuckGo`: Most recent results from [DuckDuckGo.com](https://duckduckgo.com/)
*`Instagram`: Most recent photos from an Instagram user
*`OpenClassrooms`: Lastest tutorials from [fr.openclassrooms.com](http://fr.openclassrooms.com/)
*`Pinterest`: Most recent photos from user or search
*`ScmbBridge`: Newest stories from [secouchermoinsbete.fr](http://secouchermoinsbete.fr/)
*`Wikipedia`: highlighted articles from [Wikipedia](https://wikipedia.org/) in English, German, French or Esperanto
*`Bandcamp`: Returns last release from [bandcamp](https://bandcamp.com/) for a tag
*`ThePirateBay` : Returns the newest indexed torrents from [The Pirate Bay](https://thepiratebay.se/) with keywords
*`Facebook` : Returns the latest posts on a page or profile on [Facebook](https://facebook.com/)
*`Bandcamp` : Returns last release from [bandcamp](https://bandcamp.com/) for a tag
*`Cryptome` : Returns the most recent documents from [Cryptome.org](http://cryptome.org/)
*`DansTonChat`: Most recent quotes from [danstonchat.com](http://danstonchat.com/)
*`DuckDuckGo`: Most recent results from [DuckDuckGo.com](https://duckduckgo.com/)
*`Facebook` : Returns the latest posts on a page or profile on [Facebook](https://facebook.com/)
*`FlickrExplore` : [Latest interesting images](http://www.flickr.com/explore) from Flickr
*`GoogleSearch` : Most recent results from Google Search
*`Identi.ca` : Identica user timeline (Should be compatible with other Pump.io instances)
*`Instagram`: Most recent photos from an Instagram user
*`OpenClassrooms`: Lastest tutorials from [fr.openclassrooms.com](http://fr.openclassrooms.com/)
*`Pinterest`: Most recent photos from user or search
*`ScmbBridge`: Newest stories from [secouchermoinsbete.fr](http://secouchermoinsbete.fr/)
*`ThePirateBay`: Returns the newest indexed torrents from [The Pirate Bay](https://thepiratebay.se/) with keywords
*`Twitter`: Return keyword/hashtag search or user timeline
*`Wikipedia`: highlighted articles from [Wikipedia](https://wikipedia.org/) in English, German, French or Esperanto
*`YouTube` : YouTube user channel, playlist or search
Plus [many other bridges](bridges/) to enable, thanks to the community
And [many more](bridges/), thanks to the community!
Output format
===
Output format can take several forms:
*`Atom` : ATOM Feed, for use in RSS/Feed readers
*`Mrss` : MRSS Feed, for use in RSS/Feed readers
*`Json` : Json, for consumption by other applications.
*`Html` : Simple html page.
*`Plaintext` : raw text (php object, as returned by print_r)
RSS-Bridge is capable of producing several output formats:
*`Atom` : Atom feed, for use in feed readers
*`Html` : Simple HTML page
*`Json` : JSON, for consumption by other applications
*`Mrss` : MRSS feed, for use in feed readers
*`Plaintext` : Raw text, for consumption by other applications
You can extend RSS-Bridge with your own format, using the [Format API](https://github.com/RSS-Bridge/rss-bridge/wiki/Format-API)!
- [`sqlite3`](http://php.net/manual/en/book.sqlite3.php) (only when using SQLiteCache)
Find more information on our [Wiki](https://github.com/rss-bridge/rss-bridge/wiki)
Enable / Disable bridges
===
By default, the script creates `whitelist.txt` and adds the main bridges (see above). `whitelist.txt` is ignored by git, you can edit it:
* to enable extra bridges (one bridge per line)
* to disable main bridges (remove the line)
* to enable all bridges (just one wildcard `*` as file content)
RSS-Bridge allows you to take full control over which bridges are displayed to the user. That way you can host your own RSS-Bridge service with your favorite collection of bridges!
New bridges are disabled by default, so make sure to check regularly what's new and whitelist what you want!
Find more information on the [Wiki](https://github.com/RSS-Bridge/rss-bridge/wiki/Whitelisting)
**Notice**: By default, RSS-Bridge will only show a small subset of bridges. Make sure to read up on [whitelisting](https://github.com/RSS-Bridge/rss-bridge/wiki/Whitelisting) to unlock the full potential of RSS-Bridge!
Deploy
===
Thanks to the community, hosting your own instance of RSS-Bridge is as easy as clicking a button!
[](https://my.scalingo.com/deploy?source=https://github.com/sebsauvage/rss-bridge)
[](https://heroku.com/deploy)
Getting involved
===
There are many ways for you to getting involved with RSS-Bridge. Here are a few things:
- Share RSS-Bridge with your friends (Twitter, Facebook, ..._you name it_...)
- Report broken bridges or bugs by opening [Issues](https://github.com/RSS-Bridge/rss-bridge/issues) on GitHub
- Request new features or suggest ideas (via [Issues](https://github.com/RSS-Bridge/rss-bridge/issues))
- Discuss bugs, features, ideas or [issues](https://github.com/RSS-Bridge/rss-bridge/issues)
- Add new bridges or improve the API
- Improve the [Wiki](https://github.com/RSS-Bridge/rss-bridge/wiki)
- Host an instance of RSS-Bridge for your personal use or make it available to the community :sparkling_heart:
Authors
===
We are RSS Bridge Community, a group of developers continuing the project initiated by sebsauvage, webmaster of [sebsauvage.net](http://sebsauvage.net), author of [Shaarli](http://sebsauvage.net/wiki/doku.php?id=php:shaarli) and [ZeroBin](http://sebsauvage.net/wiki/doku.php?id=php:zerobin).
Patch/contributors :
We are RSS-Bridge community, a group of developers continuing the project initiated by sebsauvage, webmaster of [sebsauvage.net](http://sebsauvage.net), author of [Shaarli](http://sebsauvage.net/wiki/doku.php?id=php:shaarli) and [ZeroBin](http://sebsauvage.net/wiki/doku.php?id=php:zerobin).
* Yves ASTIER ([Draeli](https://github.com/Draeli)) : PHP optimizations, fixes, dynamic brigde/format list with all stuff behind and extend cache system. Mail : contact /at\ yves-astier.com
Including `PHP Simple HTML DOM Parser` under the [MIT License](http://opensource.org/licenses/MIT)
The source code for RSS-Bridge is [Public Domain](UNLICENSE).
RSS-Bridge uses third party libraries with their own license:
* [`Parsedown`](https://github.com/erusev/parsedown) licensed under the [MIT License](http://opensource.org/licenses/MIT)
* [`PHP Simple HTML DOM Parser`](http://simplehtmldom.sourceforge.net/) licensed under the [MIT License](http://opensource.org/licenses/MIT)
* [`php-urljoin`](https://github.com/fluffy-critter/php-urljoin) licensed under the [MIT License](http://opensource.org/licenses/MIT)
Technical notes
===
* There is a cache so that source services won't ban you even if you hammer the rss-bridge with requests. Each bridge can have a different duration for the cache. The `cache` subdirectory will be automatically created and cached objects older than 24 hours get purged.
*To implement a new Bridge, [follow the specifications](https://github.com/RSS-Bridge/rss-bridge/wiki/Bridge-API) and take a look at existing Bridges for examples.
*To enable debug mode (disabling cache and enabling error reporting), create an empty file named `DEBUG` in the root directory (next to `index.php`).
*For more information refer to the [Wiki](https://github.com/RSS-Bridge/rss-bridge/wiki)
*RSS-Bridge uses caching to prevent services from banning your server for repeatedly updating feeds. The specific cache duration can be different between bridges. Cached files are deleted automatically after 24 hours.
*You can implement your own bridge, [following these instructions](https://github.com/RSS-Bridge/rss-bridge/wiki/Bridge-API).
*You can enable debug mode to disable caching. Find more information on the [Wiki](https://github.com/RSS-Bridge/rss-bridge/wiki/Debug-mode)
Rant
===
@@ -133,10 +261,10 @@ Rant
Your catchword is "share", but you don't want us to share. You want to keep us within your walled gardens. That's why you've been removing RSS links from webpages, hiding them deep on your website, or removed feeds entirely, replacing it with crippled or demented proprietary API. **FUCK YOU.**
You're not social when you hamper sharing by removing feeds. You're happy to have customers creating content for your ecosystem, but you don't want this content out - a content you do not even own. Google Takeout is just a gimmick. We want our data to flow, we want RSS or ATOM feeds.
You're not social when you hamper sharing by removing feeds. You're happy to have customers creating content for your ecosystem, but you don't want this content out - a content you do not even own. Google Takeout is just a gimmick. We want our data to flow, we want RSS or Atom feeds.
We want to share with friends, using open protocols: RSS, ATOM, XMPP, whatever. Because no one wants to have *your* service with *your* applications using *your* API force-feeding them. Friends must be free to choose whatever software and service they want.
We want to share with friends, using open protocols: RSS, Atom, XMPP, whatever. Because no one wants to have *your* service with *your* applications using *your* API force-feeding them. Friends must be free to choose whatever software and service they want.
We are rebuilding bridges you have wilfully destroyed.
// Time below work only with en lang. Need to think about solution. May be separate request like getFullChangelog, but to english list for all language
constDESCRIPTION="Returns list of *recent* torrents for a specific show on EZTV. Get showID from URLs in https://eztv.ch/shows/showID/show-full-name.";
constMAINTAINER='alexAubin';
constNAME='EZTV';
constURI='https://eztv.ch/';
constDESCRIPTION='Returns list of *recent* torrents for a specific show
on EZTV. Get showID from URLs in https://eztv.ch/shows/showID/show-full-name.';
constPARAMETERS=array(array(
'i'=>array(
'name'=>'Show ids',
'exampleValue'=>'showID1,showID2,…',
'required'=>true
'i'=>array(
'name'=>'Show ids',
'exampleValue'=>'showID1,showID2,…',
'required'=>true
)
));
@@ -22,38 +23,38 @@ class EZTVBridge extends BridgeAbstract{
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.