Personally, I've never been a fan of pagination.
Asking users to paginate through content or to request more by clicking on a button works, but it's not very elegant.
Lazy loading content provides a more fluid user experience and increases the resposiveness of a page by only loading content when it's required. It's also a great way to keep users engaged as new content just "appears" once they reach a certain point in the page.
After some quick experimentation with the JavaScript IntersectionObserver, we were up and running with a lazy loading page within about 30 lines of code.
To be clear, this isn't "virtual scrolling" which can be achieved with the IntersectionObserver, however it's about half way there with the exception of not removing content from the dom.
The example
We're going to create a mock database of messages, featuring an index, title and some content.
On loading the page, the IntersectionObserver will trigger a function to request 10 messages which will then be rendered to the dom. Once the user reaches the bottom of those 10 messages, the function will trigger again and repeat the process until we're out of messages.
Server side
We're using Python & the Flask framework in this example but the same principles apply to any language or web server.
Python
from flask import Flask, render_template, request, jsonify, make_response
import random
import time
app = Flask(__name__)
heading = "Lorem ipsum dolor sit amet."
content = """
Lorem ipsum dolor sit amet consectetur, adipisicing elit.
Repellat inventore assumenda laboriosam,
obcaecati saepe pariatur atque est? Quam, molestias nisi.
"""
db = list() # The mock database
posts = 500 # num posts to generate
quantity = 20 # num posts to return per request
for x in range(posts):
"""
Creates messages/posts by shuffling the heading & content
to create random strings & appends to the db
"""
heading_parts = heading.split(" ")
random.shuffle(heading_parts)
content_parts = content.split(" ")
random.shuffle(content_parts)
db.append([x, " ".join(heading_parts), " ".join(content_parts)])
@app.route("/")
def index():
""" Route to render the HTML """
return render_template("index.html")
@app.route("/load")
def load():
""" Route to return the posts """
time.sleep(0.2) # Used to simulate delay
if request.args:
counter = int(request.args.get("c")) # The 'counter' value sent in the QS
if counter == 0:
print(f"Returning posts 0 to {quantity}")
# Slice 0 -> quantity from the db
res = make_response(jsonify(db[0: quantity]), 200)
elif counter == posts:
print("No more posts")
res = make_response(jsonify({}), 200)
else:
print(f"Returning posts {counter} to {counter + quantity}")
# Slice counter -> quantity from the db
res = make_response(jsonify(db[counter: counter + quantity]), 200)
return res
The HTML
We're using the animate.css library for some simple animations when new messages come into view. We're also using the Bootstrap 4 library.
You'll also note the use of the HTML <template> tag.
HTML
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<!-- Bootstrap CSS -->
<link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css" integrity="sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T" crossorigin="anonymous">
<!-- Animate CSS -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/animate.css/3.7.0/animate.min.css">
<title>Infinite load</title>
</head>
<!-- sticky-top navbar displaying the counter -->
<nav class="navbar navbar-light bg-light sticky-top">
<div class="container">
<a class="navbar-brand" id="loaded" href="#">0 items loaded</a>
</div>
</nav>
<body>
<main class="mt-3">
<div class="container">
<div class="row">
<div class="col">
<h1 class="display-4 mb-3">Infinite load</h1>
<!-- div to contain the content -->
<div id="scroller" class="mb-3">
<!-- template schema, hidden from the dom -->
<template id="post_template">
<!-- template content -->
<div class="card mb-3 animated fadeIn shadow-sm">
<div class="card-body">
<h4 class="card-title" id="title"></h4>
<span class="text-muted" id="content"></span>
</div>
</div>
</template>
</div>
<!-- element to trigger the IntersectionObserver -->
<div class="d-flex justify-content-center mb-3" id="sentinel">
<div class="spinner-border" role="status"></div>
</div>
</div>
</div>
</div>
</main>
<!-- Import Bootstrap JS here -->
</body>
</html>
The key point to note in the HTML:
HTML
<div class="d-flex justify-content-center mb-3" id="sentinel">
<div class="spinner-border" role="status"></div>
</div>
The sentinel div will be watched by the IntersectionObserver and will trigger a new request for data when it comes into the window view.
We've placed it at the bottom of the scroller div so when the user reaches the last of the messages, more content is loaded and appended to the dom.
The JavaScript
JavaScript
// Get references to the dom elements
var scroller = document.querySelector("#scroller");
var template = document.querySelector('#post_template');
var loaded = document.querySelector("#loaded");
var sentinel = document.querySelector('#sentinel');
// Set a counter to count the items loaded
var counter = 0;
// Function to request new items and render to the dom
function loadItems() {
// Use fetch to request data and pass the counter value in the QS
fetch(`/load?c=${counter}`).then((response) => {
// Convert the response data to JSON
response.json().then((data) => {
// If empty JSON, exit the function
if (!data.length) {
// Replace the spinner with "No more posts"
sentinel.innerHTML = "No more posts";
return;
}
// Iterate over the items in the response
for (var i = 0; i < data.length; i++) {
// Clone the HTML template
let template_clone = template.content.cloneNode(true);
// Query & update the template content
template_clone.querySelector("#title").innerHTML = `${data[i][0]}: ${data[i][1]}`;
template_clone.querySelector("#content").innerHTML = data[i][2];
// Append template to dom
scroller.appendChild(template_clone);
// Increment the counter
counter += 1;
// Update the counter in the navbar
loaded.innerText = `${counter} items loaded`;
}
})
})
}
// Create a new IntersectionObserver instance
var intersectionObserver = new IntersectionObserver(entries => {
// Uncomment below to see the entry.intersectionRatio when
// the sentinel comes into view
// entries.forEach(entry => {
// console.log(entry.intersectionRatio);
// })
// If intersectionRatio is 0, the sentinel is out of view
// and we don't need to do anything. Exit the function
if (entries[0].intersectionRatio <= 0) {
return;
}
// Call the loadItems function
loadItems();
});
// Instruct the IntersectionObserver to watch the sentinel
intersectionObserver.observe(sentinel);
IntersectionObserver has many configuartion option which you can read about here over at the Mozilla developer docs.
In this example, we're calling the loadItems function if the intersectionRatio of the first entry is greater than 0. Which basically means "Do nothing if the sentinel element is not in view".
Uncomment the console.log(entry.intersectionRatio); section and scroll down to the bottom of the page to load more content. You'll see the values printed in the developer tools.
Wrapping up
After initial testing and playing around with the IntersectionObserver, we found it's easy to work with and setup an infitite, lazy loading feed of new content with very little code.
Next steps & testing will be to build a true virtual scrolling machanism, which dynamically inserts and removes elements from the dom depending on what's currently observable in the window.
Related items
Building scalable Flask applications from the start using the application factory pattern, blueprints and the current_app proxy
Using HTTP request methods as the primary means of access control
Debugging and connecting to an Azure Cosmos DB using the MongoDB API using PyMongo, PyMODM and MongoEngine
Harnessing the power of Pythons enumerations and exploring the enum module