Django + REST Framework: Pagination, Filtering, and Search Made Simple

Building robust APIs often requires more than just basic CRUD operations. Users expect to search through data, filter results, and navigate large datasets efficiently. Django REST Framework (DRF) provides powerful built-in tools to handle pagination, filtering, and search functionality with minimal code. In this post, we'll explore how to implement these features effectively.

Setting Up the Foundation

Before diving into advanced features, let's establish a basic model and serializer to work with:

# models.py
from django.db import models

class Article(models.Model):
    title = models.CharField(max_length=200)
    content = models.TextField()
    author = models.CharField(max_length=100)
    category = models.CharField(max_length=50)
    published_date = models.DateTimeField(auto_now_add=True)
    is_published = models.BooleanField(default=True)
    views = models.PositiveIntegerField(default=0)

    def __str__(self):
        return self.title

# serializers.py
from rest_framework import serializers
from .models import Article

class ArticleSerializer(serializers.ModelSerializer):
    class Meta:
        model = Article
        fields = '__all__'

Implementing Pagination

Pagination prevents overwhelming clients with massive datasets and improves API performance. DRF offers several pagination styles out of the box.

Global Pagination Settings

First, configure pagination globally in your settings.py:

# settings.py
REST_FRAMEWORK = {
    'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.PageNumberPagination',
    'PAGE_SIZE': 20
}

Custom Pagination Classes

For more control, create custom pagination classes:

# pagination.py
from rest_framework.pagination import PageNumberPagination, LimitOffsetPagination
from rest_framework.response import Response

class CustomPageNumberPagination(PageNumberPagination):
    page_size = 10
    page_size_query_param = 'page_size'
    max_page_size = 100
    
    def get_paginated_response(self, data):
        return Response({
            'pagination': {
                'next': self.get_next_link(),
                'previous': self.get_previous_link(),
                'count': self.page.paginator.count,
                'page_size': self.page_size,
                'current_page': self.page.number,
                'total_pages': self.page.paginator.num_pages,
            },
            'results': data
        })

class CustomLimitOffsetPagination(LimitOffsetPagination):
    default_limit = 10
    limit_query_param = 'limit'
    offset_query_param = 'offset'
    max_limit = 100

Using Pagination in Views

# views.py
from rest_framework.generics import ListAPIView
from .models import Article
from .serializers import ArticleSerializer
from .pagination import CustomPageNumberPagination

class ArticleListView(ListAPIView):
    queryset = Article.objects.all()
    serializer_class = ArticleSerializer
    pagination_class = CustomPageNumberPagination

Adding Filtering Capabilities

Filtering allows users to narrow down results based on specific criteria. DRF integrates seamlessly with django-filter for advanced filtering options.

Installing and Configuring django-filter

pip install django-filter

Add it to your INSTALLED_APPS and configure DRF:

# settings.py
INSTALLED_APPS = [
    # ... other apps
    'django_filters',
]

REST_FRAMEWORK = {
    'DEFAULT_FILTER_BACKENDS': [
        'django_filters.rest_framework.DjangoFilterBackend',
        'rest_framework.filters.SearchFilter',
        'rest_framework.filters.OrderingFilter',
    ],
    # ... other settings
}

Basic Filtering

# views.py
from django_filters.rest_framework import DjangoFilterBackend
from rest_framework.filters import SearchFilter, OrderingFilter

class ArticleListView(ListAPIView):
    queryset = Article.objects.all()
    serializer_class = ArticleSerializer
    pagination_class = CustomPageNumberPagination
    filter_backends = [DjangoFilterBackend, SearchFilter, OrderingFilter]
    
    # Simple field filtering
    filterset_fields = ['category', 'author', 'is_published']
    
    # Search functionality
    search_fields = ['title', 'content', 'author']
    
    # Ordering options
    ordering_fields = ['published_date', 'views', 'title']
    ordering = ['-published_date']  # Default ordering

Advanced Filtering with FilterSets

For more complex filtering logic, create custom FilterSet classes:

# filters.py
import django_filters
from .models import Article

class ArticleFilter(django_filters.FilterSet):
    title = django_filters.CharFilter(lookup_expr='icontains')
    author = django_filters.CharFilter(lookup_expr='iexact')
    category = django_filters.ChoiceFilter(choices=[
        ('tech', 'Technology'),
        ('science', 'Science'),
        ('business', 'Business'),
    ])
    published_after = django_filters.DateTimeFilter(
        field_name='published_date', 
        lookup_expr='gte'
    )
    published_before = django_filters.DateTimeFilter(
        field_name='published_date', 
        lookup_expr='lte'
    )
    min_views = django_filters.NumberFilter(
        field_name='views', 
        lookup_expr='gte'
    )
    
    class Meta:
        model = Article
        fields = []

Update your view to use the custom filter:

# views.py
class ArticleListView(ListAPIView):
    queryset = Article.objects.all()
    serializer_class = ArticleSerializer
    pagination_class = CustomPageNumberPagination
    filter_backends = [DjangoFilterBackend, SearchFilter, OrderingFilter]
    filterset_class = ArticleFilter
    search_fields = ['title', 'content', 'author']
    ordering_fields = ['published_date', 'views', 'title']
    ordering = ['-published_date']

Implementing Search Functionality

DRF's built-in SearchFilter provides powerful search capabilities across multiple fields.

Basic Search Configuration

class ArticleListView(ListAPIView):
    # ... other configurations
    search_fields = [
        'title',           # Exact field search
        'content',         # Search in content
        '=author',         # Exact match for author
        '^title',          # Search from the beginning of title
        '@content',        # Full-text search (PostgreSQL only)
    ]

Custom Search Implementation

For more advanced search logic, override the search functionality:

# views.py
from django.db.models import Q

class ArticleListView(ListAPIView):
    queryset = Article.objects.all()
    serializer_class = ArticleSerializer
    pagination_class = CustomPageNumberPagination
    filter_backends = [DjangoFilterBackend, OrderingFilter]
    filterset_class = ArticleFilter
    ordering_fields = ['published_date', 'views', 'title']
    ordering = ['-published_date']
    
    def get_queryset(self):
        queryset = super().get_queryset()
        search_query = self.request.query_params.get('search', None)
        
        if search_query:
            queryset = queryset.filter(
                Q(title__icontains=search_query) |
                Q(content__icontains=search_query) |
                Q(author__icontains=search_query)
            ).distinct()
        
        return queryset

Combining Everything Together

Here's a complete example that combines pagination, filtering, and search:

# views.py
from rest_framework.generics import ListAPIView
from rest_framework.response import Response
from django_filters.rest_framework import DjangoFilterBackend
from rest_framework.filters import SearchFilter, OrderingFilter
from .models import Article
from .serializers import ArticleSerializer
from .filters import ArticleFilter
from .pagination import CustomPageNumberPagination

class ArticleListView(ListAPIView):
    queryset = Article.objects.select_related().prefetch_related()
    serializer_class = ArticleSerializer
    pagination_class = CustomPageNumberPagination
    filter_backends = [DjangoFilterBackend, SearchFilter, OrderingFilter]
    filterset_class = ArticleFilter
    search_fields = ['title', 'content', 'author']
    ordering_fields = ['published_date', 'views', 'title']
    ordering = ['-published_date']
    
    def list(self, request, *args, **kwargs):
        # Add custom logic before processing the request
        queryset = self.filter_queryset(self.get_queryset())
        
        # Add metadata to response
        page = self.paginate_queryset(queryset)
        if page is not None:
            serializer = self.get_serializer(page, many=True)
            response = self.get_paginated_response(serializer.data)
            
            # Add metadata
            response.data['filters_applied'] = bool(request.query_params)
            response.data['total_results'] = queryset.count()
            
            return response
        
        serializer = self.get_serializer(queryset, many=True)
        return Response(serializer.data)

API Usage Examples

With the implementation above, your API supports various query patterns:

Pagination

GET /api/articles/?page=2&page_size=15

Filtering

GET /api/articles/?category=tech&author=john&min_views=100
GET /api/articles/?published_after=2023-01-01&is_published=true

Search

GET /api/articles/?search=django tutorial

Ordering

GET /api/articles/?ordering=-views,title

Combined

GET /api/articles/?search=python&category=tech&ordering=-published_date&page=1&page_size=10

Performance Optimization Tips

1. Database Indexing: Add indexes on frequently filtered fields:

class Article(models.Model):
    category = models.CharField(max_length=50, db_index=True)
    published_date = models.DateTimeField(auto_now_add=True, db_index=True)
    # ... other fields

2. Query Optimization: Use select_related() and prefetch_related():

queryset = Article.objects.select_related('author').prefetch_related('tags')

3. Limit Search Fields: Don't search on large text fields unless necessary.

4. Cache Common Queries: Use Django's caching framework for frequently accessed data.

Conclusion

Django REST Framework makes implementing pagination, filtering, and search functionality straightforward with its built-in tools and extensive customization options. By combining these features thoughtfully, you can create APIs that are both powerful and performant.

The key is to start simple with the default implementations and gradually customize based on your specific requirements. Remember to consider performance implications, especially when dealing with large datasets, and always test your API endpoints thoroughly with realistic data volumes.

With these tools in your toolkit, you can build APIs that provide excellent user experiences while maintaining clean, maintainable code.