Problem Solving

Problems you may come across and tips on how to solve them.

Scripts in pulled content are not executing

The most likely reason for scripts simply not executing is that they are sanitized out of the pulled content. Have a look at the Advanced tab Sanitize and Niceify section and make sure Sanitize output is set to none.

Beyond that, further problems are typically related to.

  1. AssetsUniversal Content Puller will load required assets for sources inside the site. However, UCP knows nothing about assets for external sources. You may need to explicitly load assets for extrenal sources using a header extra content attribute. Take care when doing so, pulling in assets for external sources can create security issues.
  2. Race - The JavaScript runs before its container is ready. The solution is usually to delay initialising until the container is ready for it, such as the when visble trick used to make maps work.
  3. Wait - The JavaScript is waiting for an initialisation such as a 'ready' event that has already happened. The solution is usually to trigger the event, though take care that some JavaScript may not like being initialised twice.

Do indexes start at 0 or 1?

In most cases, Universal Content Puller indexes start from 1. For example, rows and columns in a table or list. This is because it is most conveniently user-friendly.

The exception is anywhere that something is sliced, where the first item has an index of 0. So slicing blocks in an area or items in a list start at 0. This is because slicing follows php rules, including negative slicing from the end of a list. To change those rules could be even more confusing than an index that starts at 0.

 

Autolinking corrupts HTML

Autolinking and sanitization are applied on the output of the content transform, before the content display begins to format the transformed content for display. If the transformed content is an array, the autolinking and/or sanitization is applied on each element.

Universal Content Puller provides two alternatives for autolinking. 

  1. The core autlinker only works with plain text and fully specified URLs beginning http://... If you let the core autolinker loose on content that is already html it will corrupt attributes such a ssrc and href.
  2. The advanced autolinker is more sophisticated and can work with content that is already html. It is also more capable of picking up abbreviated URLS such as www.mydomain.com... and URLs with query strings. Nevertheless, it is not foolproof. Apply it to content that mixes complex html and javascript and it may try and autolink something it isn't supposed to.

Overall, any complex html content should already have links in it and should not require autololinking.

Images appear as URLs, not as images

If a content source specifies a URL or path to an image without actually making that an html image element, it will by deafult appear as a URL.

To show it as an image, you need to enable the advanced autolinker and then select to Render image URLs as images. Converting image URLs to images is only available with the advanced autolinker.

How do I style pulled content?

Universal Content Puller by default wraps all pulled content in  div and ucp-body. Within complex content, each item in a list or table is also assigned a class by UCP.

The first option for styling is in the Advanced tab of the edit dialog, where you can change the wrapper element and class. For example:

  • Change the wrapper to blockquote and ucp-body to blockquote pulled content.
  • Chnage the wrapper to div and alert alert-info to put it into a bootstrap info box - assuming your theme is bootstrap based.

Beyond that, you can declare styles for the classes UCP has added to your site theme or to a header extra content attribute, as demonstrated in Example - RSS feed.

Further content sources, transforms and displays

Universal Content Puller is a growing framework for adding further Content Sources, Content Transforms and Content Displays.

If you can't find what you need, please ask me and explain your requirement. It may be that you just need a few tips on how to do it with existing plugins.

I have plans to add further sources, transforms and displays to UCP. Please chack these linked documentation pages for the latest list - other documentation may lag behind them.

The pluggable architecture means that as well as extension withing UCP, third party packages and /application/ classes can be used to add pluggins. This could be a s simple as copying and modifying a display to add your own formatting to creating a completely new kind of display. As long as it follows the plugin architecture and hass a unique plugin name, you can create as many custom plugins as you need. Please contact me for advice.

I am also available for contract development of UCP plugins if you need me. A UCP plugin will usually be considerably cheaper to develop than a full bespoke interface addon.

AJAX loading continues to show the placeholder

When Universal Content Puller is configured to AJAX load the pulled content (in the UCP edit dialog Advanced tab), a placeholder is shown until the AJAX load is complete.

If the placeholder continues to be shown:

  1. Make sure the UCP block is not cached. See the Cache settings in the UCP edit dialog Advanced tab.
  2. Make sure no form of Sanitize output is enabled in the UCP edit dialog Advanced tab. Sanitzation could strip necessary JavaScript or data attributes needed for the AJAX loading and pagination to work.
  3. Check the AJAX load is actually happening in your browser developer console, Network tab, XHR requests.
  4. Enable the Debug setting at the bottom of the UCP edit dialog Advanced tab and check the debug trace in your browser developer console. 

Pagination with more than one UCP block on the page

If you have more than one UCP block on a page requiring pagination, you have some choices on how that behaves:

  1. Leave the block identity out of the pagination. All UCP blocks will then paginate to the same page. Sometimes this can be what you want, like all the RSS Feed examples paginating to gether in Eaxmple - RSS Feed.
  2. Include the block identity in the pagination. In this case, when one UCP block is paginated, other UCP blocks on the page will go back to their first page.
  3. Set the UCP blocks to AJAX load. When a UCP block is AJAX loaded, any pagination for that block is also AJAX loaded. So all blocks can paginate by AJAX independantly.

Can UCP break my site?

Universal Content Puller won't break your site directly, but the content you pull could break the page it is pulled into.

Infinite Recursion

UCP includes built in recursion detection. Nevertheless, if you try hard enough you could create a recursive loop that pulls content within content within content... You will need to tray hard to do such, but it is theoretically possible.

Dangerous JavaScript

When pulling content from an external source, or any source you are not confident in, it is always best to start with Sanitize output set to Safe in the Advanced tab of the edit dialog. It is set that way in the default installation and unless you have a good reason to change it, is best left that way so you don't forget. JavaScript will be removed from the pulled content.

However,the converse is that where pulled content contains JavaScript that is required to execute, it won't do so unless you set Sanitize output set to None - see above.

Broken HTML

Just like an HTML block containing mis-matched or otherwise broken HTML tags could break the page it is placed on, if you pull content containing such broken tags it could break the page it is pulled into.

Imported Settings

You may have noticed that many of the Universal Content Puller examples have small button 'View settings' that pops up the settings used in that block as JSON data. You can copy and paste those settings into your own UCP blocks as a starting point for your own projects. But take care, the settings can often include things like concrete5 Page IDs or File IDs that will you will need to correct before saving the edit dialog to work with your site.

Solutions

Edit mode marker

By default UCP renders a marker in edit mode rather than the pulled content. This is so any broken content you pull cannot break concrete5 in edit mode.If you can get a page into edit mode, you can resolve any problems that break the front end view.

Page versions

If you can't get a page into edit mode, you should be able to go directly to the sitemap in the dashboard at yoursite.com/index.php/dashboard/sitemap. Once in the sitemap, click on a page to get the popdown menu and select versions. Once in versions, approve a previous page version and then delete the erroneous page version.

Panic mode

If all else fails, you can set UCP into Panic mode by using FTP or SSH to edit /application/config/generated_overrides/jl_universal_content_puller.php and set 'panic' => true. When panic is set, UCP will render all blocks as markers, even outside of edit mode. This will enable you to edit the problem block in concrete5.

Serialize

If you are unsure of content, in any UCP block you can set the Content Display to Serialize. This will output any pulled content as serialized data, so you can study it and head off any issues before committing yourself to displaying faulty content.

Additional Pages

About this Sidebar

Creating a sidebar for a group of pages without messing about with stacks is an easy use-case for Universal Content Puller.

This sidebar is edited once, within the main addon page for Universal Content Puller.

It is then pulled into all UCP sub-pages using a UCP block.

The Content Source is Parent Page, set to pull the Sidebar area from 2 pages from the top. The Content Transform is Selector, set to remove container and row classes that, when unnecessarily nested, could mess up the Bootstrap grid. The Content Display is Plain, which just outputs the transformed text.

In the advanced settings, sanitization is disabled as we trust the source page and don't want to strip out any formatting or functionality from the pulled sidebar.