Advanced

The Advanced tab of the Universal Content Puller edit dialog provides further options common to all UCP source/transform/display workflows.

The Support tab provides sources of further support and also useful Export and Import tools for UCP settings.

AJAX

There are two broad reasons for AJAX loading pulled content. Performance and Interaction.

By default, UCP loads pulled content directly into a page. The alternative is to delay pulling the content until after the page has loaded into the browser and then load it by AJAX. For example, a content source could be a little slow responding or so complex or image heavy that it creates an unwanted delay in the web server.

An example of interaction would be to only load content in response to a user interaction elsewhere on a page. For example, a user clicks a button for more details and the detailed content is then loaded.

UCP supports variations of both of these use-cases.

Load with page - no ajax

Loading directly with the page, no AJAX, is the default setting for a very good reason.

In many simple cases the AJAX overhead of placing a marker element in the page and the relevant JavaScript to then load the actual content is just as high an overhead as actually rendering the pulled content directly. Considered use of block caching can further reduce the need for AJAX loading.

Loading pulled content directly is also least likely to result in issues with JavaScript used in the pulled content.

Ajax load immediately

The pulled content is loaded immediately the page has rendered in the browser.

Ajax load after delay

After the page has rendered in the browser there is a configurable delay before the the pulled content is loaded. A delay of 0ms is the equivalent of loading immediately.

Ajax load when visible

There is a subtle difference between proximity to viewport and visibility. A pulled content container could be visible but outside the viewport, or hidden even when inside the viewport. For example, a set of tabs or accordions could be inside the viewport, but all except the first tab or accordion panel would not be visible until that tab or accordion is opened.

When the pulled content container becomed visible, the pulled content is AJAX loaded.

Ajax load when scrolled into viewport - lazy load

Proximity is adjusted by a setting a proximity in pixels. When the browser is scrolled or resized to bring the container for the pulled content within that proximity of the viewport the pulled content is loaded.

As a guideline, allow about half the anticipated window height and adjust that depending on how long you expect ajax loading the pulled content to take. To present a seamless experience to users, you usually want pulled content to begin loading sufficiently far ahead in the page scrolling to have completed by the time the user gets to it!

A positive proximity is distance outside the viewport. A negative proximity is distance inside the viewport.

Ajax load when in viewport

Loads only when in viewport, or close by proximity, both horizontally and vertically. Proximity is polled, so works with lazy loaded elements and any other dynamic manipulation of content.

A positive proximity is distance outside the viewport. A negative proximity is distance inside the viewport.

Ajax load when in viewport and visible

Combines proximity and visibility, so content is only loaded when both visible and inside the viewport.

A positive proximity is distance outside the viewport. A negative proximity is distance inside the viewport.

Ajax load when placeholder clicked

UCP provides a configurable placeholder that is shown until the pulled content is AJAX loaded. This setting turns that placeholder into a clickable element, loading the pulled content when it is clicked.

Placeholder options are

• No visible placeholder - so not much use with this AJAX option.
• Blank area - with optional label.
• Outlne area - provides a faint outline with an optional label.
• Default - a Bootstrap .well with a label.

The label and height of the placeholder are also configurable.

You can further style the placeholder with CSS targetting the class .ucp-ajax-placeholder

Ajax load on ucp_load event

This is the setting for building more complex interactions. The pulled content will only be loaded when the container receives the JavaScript event ucp_load. It is up to the site developer to provide JavaScript to trigger that event, such as by attaching a handler to a link or button which then triggers the ucp_load event.

A further event, ucp_loaded, is triggered by UCP whenever content has been successfully loaded by AJAX. ucp_loaded is always triggered, irrespective of the AJAX option.

Cache

Universal Content Puller supports options for caching in all stages of the content pulling process. With the Content Source, with the Content Transform, and with the block cache on Content Display. Then Concrete CMS can cache an entire page.

Whilst this support page is primarily about the Advanced tab of the edit dialogue, here we digress into the levels of cache available.

It is definitely not a good idea to configure caching at all these levels! In a worst case, the cumulative overhead of managing multiple levels of cache could be worse than no cache at all!

For any particular UCP usage, review the cache options available and apply caching where it provides the greatest advantage.

Cache settings can be set specifically to a UCP instance, or configured to track the equivalent Concrete CMS page or block cache settings.

After changing cache settings, remember to check the page both immediately after publishing and later, while logged out. You don't want an over-indulgant cache setting revealing content that should not be available to casual visitors or showing an edit mode placeholder.

Be careful with cache settings if pagination is enabled. Caching is usually best inserted before pagination.

Source Cache

Some content sources, such as pulling from a URL, can cache the source incoming to UCP. This is useful when, for example reading an external source that may be slow and doesn't change too often. For example, many of our examples reading RSS sources or CSV files directly from the web use source cacheing to avoid repeat queries to such external sources.

Transform  Cache

Cacheing in the Content Transform stage is facilitated by a dedicated Cache transform. You can insert this at any stage of a transform pipeline to cache the incoming data at that stage of the pipeline.

Block Cache

Rather than heading off to the settings available in block edit menu > Advanced. You can conveniently manage block cache settings directly in the UCP Advanced tab. 

Choice of block cache settings depends greatly on what content is pulled. For example, if pulling content from a bulky or slow external URL that doesn't change often, you may want to enable all options and set the cache for a long lifetime. On the other hand, if you are pulling data that changes dynamically and need it to be up-to-date, you would likely only want a shortt cache lifgetime or to leave it disabled.

Sanitize and Niceify

Sanitize

Pulled content can contain styling and JavaScript that you do not want to execute on your page. There could be:

• JavaScript you trust, but just don't want.
• JavaScript that would break if executed in the context of the page that pulls it.
• A source where you don't have control over the content or security, so could be used by a third party to attack the security of your site.

Sanitization can be selected at settings:

• None - we trust it and maybe even want any JavaScript to execute.
• Escape - the pulled content will be escaped, so any html elements and JavaScript will be rendered in a visible form on the page and not executed.
• Safe - Remove JavaScript elements and styles
• Safe and Styled - As safe, but allowing syles.
• Basic elements - Only allowing elements  p, br, strong, em, strike, a

Nicify

The converse of sanitizing is that content may contain URLs for other pages or images that are not already rendered as clickable links of shown as images.

Autolinking is the process of taking text URLs found in the page and turning them into clickable links. Autolinking is not a perfect process, so UCP provides multiple mechanisms for autolinking:

1. The ConcreteCMS core autolinker. This works best with plain text. When applied to HTML it will corrupt existing links.
2. A modification of the ConcreteCMS core autolinker that handles '//' protocol and renders image-like URLs as images. As with the core ConcreteCMS autolinker this works best with plain text. When applied to HTML it will corrupt existing links.
3. An advanced autolinker. This works with plain text and with sanitized html, but may still corrupt more complex html and URLs in JavaScript in the pulled content.

All autolinkers can be configured to link to the same window/tab, or to a _blank tab.

When the modified core autolinker or advanced autolinker are selected, a further option is to render image URLs as either links to URLs or as image elements to show the image, using any configured ConcreteCMS thumbnail size. Beware that thumbnail sizes are for sizing compatible with the site theme and do not actually resize the linked image file. External images are not resized and may be very big.

Wrapper

By default, UCP wraps all pulled content with <div class="ucp-body"></div>.

The advanced options for the wrapper allow the wrapping element and/or class to be changed, or even to disable wrapping altogether. When there is a wrapping element, UCP will always ensure the classes rendered include the class 'ucp-body'

For example, you could change the wrapping element to a <blockquote class="my-quote-class ucp-body"></div>

Developer

Here we have settings that are best left alone unless you know what you are doing. 

Edit mode marker

UCP will normally only show a marker when in edit mode without actually pulling content, so that nothing in the pulled content can interfere with the ability to edit the page.

The edit mode marker includes some basic information about the configured content source, transform and display.

If you are confident that content will not interfere in any way, you can turn off the edit mode marker and get a clearer view of what is being pulled.

Recursion message

When pulling content that itself pulls content, you could inadvertently create a loop that continues in ever continuing cirlces of recursion that eventuslly cause a server error. 

UCP includes built in recursion protection and will report when recursion is detected with an error message on the page. If you don't want vistors to see that, you can turn it off the message. Recursion will still be detected and logged, but will not be shown on the page.

Nevertheless, if you try hard enough you may be able to create a convoluted loop of recursion that UCP is unable to detect and that will subsequently lead to a broken page.

If that happens, you may

1. Revert the page version from the dashboard sitemap.
2. Set 'panic' to true in the UCP config settings at application/config/generated_overrides/jl_universal_content_puller.php. This will render all UCP blocks as markers while you edit the offending block and fix the problem.

Debug mode

The debug mode setting will result in UCP outputting JavaScript information to the browser developer console, particularly when AJAX loading is enabled. There may also be some on-page information shown by UCP plugins.

Support Tab

Import and Export

The Support tab of the UCP edit dialog provides buttons to Export and Import settings. Settings are exported as JSON data which can then be imported to other UCP blocks.

To some extent, Export and Import can even be used to copy UCP settings between sites, though take care as any page or file IDs in the JSON may not map accurately between sites and will need editing for the desired copy to work.

As well as simply copying and pasting the JSON as text, the settings can also be downloaded to and uploaded from files, or stored with a name in the Concrete database. When stored in the database, the name must be at least 5 characters long.

Further documentation and support

Also in the support tab are links and advice on obtaining further support for Universal Content Puller.