Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support for django rest framework 3.0 #10

Merged
merged 4 commits into from
Mar 12, 2015
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,15 @@ python:
- "2.7"
- "3.4"
env:
- DJANGO="django==1.6" REST="djangorestframework==2.4.4" PANDAS="pandas==0.15.1"
- DJANGO="django==1.7" REST="djangorestframework==2.4.4" PANDAS="pandas==0.15.1"
matrix:
- DJANGO="django==1.6.10" REST="djangorestframework==2.4.4"
- DJANGO="django==1.6.10" REST="djangorestframework==3.1.0"
- DJANGO="django==1.7.6" REST="djangorestframework==2.4.4"
- DJANGO="django==1.7.6" REST="djangorestframework==3.1.0"
global:
- PANDAS="pandas==0.15.2"
install:
- pip install requests
- pip install $DJANGO
- pip install $REST
- pip install $PANDAS
Expand Down
29 changes: 18 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,31 +66,38 @@ The example below assumes you already have a Django project set up with a single
from rest_pandas import PandasView
from .models import TimeSeries
class TimeSeriesView(PandasView):
# Django REST Framework 2.4
model = TimeSeries

# Django REST Framework 3+
queryset = TimeSeries.objects.all()
serializer_class = TimeSeriesSerializer

# In response to get(), the underlying Django REST Framework ListAPIView
# will load the default queryset (self.model.objects.all()) and then pass
# it to the following function.
# will load the queryset and then pass it to the following function.

def filter_queryset(self, qs):
# At this point, you can filter queryset based on self.request or other
# settings (useful for limiting memory usage)
return qs

# Then, the included PandasSerializer will serialize the queryset into a
# simple list of dicts (using the DRF ModelSerializer). To customize
# which fields to include, subclass PandasSerializer and set the
# appropriate ModelSerializer options. Then, set the serializer_class
# property on the view to your PandasSerializer subclass.
# Then, the default serializer (typically a DRF ModelSerializer) should
# serialize each row in the queryset into a simple dict format. To
# customize which fields to include, create a subclass of ModelSerializer
# and assign it to serializer_class on your view.

# Next, the included PandasSerializer will load the ModelSerializer result
# into a DataFrame and pass it to the following function on the view.

# Next, the PandasSerializer will load the ModelSerializer result into a
# DataFrame and pass it to the following function on the view.

def transform_dataframe(self, dataframe):
# Here you can transform the dataframe based on self.request
# (useful for pivoting or computing statistics)
return dataframe


# For more control over dataframe creation, subclass PandasSerializer and
# set pandas_serializer_class on the view. (Or set list_serializer_class
# on your ModelSerializer subclass' Meta class if you're using DRF 3).

# Finally, the included Renderers will process the dataframe into one of
# the output formats below.
```
Expand Down
62 changes: 38 additions & 24 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
Django REST Pandas
==================

*Django REST Framework + pandas = A Model-driven Visualization API*
`Django REST Framework <http://django-rest-framework.org>`__ + `pandas <http://pandas.pydata.org>`__ = A Model-driven Visualization API
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

**Django REST Pandas** (DRP) provides a simple way to generate and serve
`pandas <http://pandas.pydata.org>`__ DataFrames via the `Django REST
Framework <http://django-rest-framework.org>`__. The resulting API can
serve up CSV (and a number of other formats)
serve up CSV (and a number of `other formats <#supported-formats>`__)
for consumption by a client-side visualization tool like
`d3.js <http://d3js.org>`__.

Expand Down Expand Up @@ -94,17 +95,23 @@ Framework. This means clients can specify a format via
``Accepts: text/csv`` or by appending ``.csv`` to the URL (if the URL
configuration below is used).

.. csv-table::
:header: "Format", "Content Type", "pandas Dataframe Function", "Notes"
:widths: 50, 150, 70, 500

CSV,``text/csv``,``to_csv()``,
TXT,``text/plain``,``to_csv()``,"Useful for testing, as most browsers will download a CSV file instead of displaying it"
JSON,``application/json``,``to_json()``,
XLSX,``application/vnd.openxml...sheet``,``to_excel()``,
XLS,``application/vnd.ms-excel``,``to_excel()``,
PNG,``image/png``,``plot()``,"Currently not very customizable, but a simple way to view the data as an image."
SVG,``image/svg``,``plot()``,"Eventually these could become a fallback for clients that can't handle d3.js"
+----------+---------------------------------------+-----------------------------+------------------------------------------------------------------------------------------+
| Format | Content Type | pandas DataFrame Function | Notes |
+==========+=======================================+=============================+==========================================================================================+
| CSV | ``text/csv`` | ``to_csv()`` |
+----------+---------------------------------------+-----------------------------+------------------------------------------------------------------------------------------+
| TXT | ``text/plain`` | ``to_csv()`` | Useful for testing, as most browsers will download a CSV file instead of displaying it |
+----------+---------------------------------------+-----------------------------+------------------------------------------------------------------------------------------+
| JSON | ``application/json`` | ``to_json()`` |
+----------+---------------------------------------+-----------------------------+------------------------------------------------------------------------------------------+
| XLSX | ``application/vnd.openxml...sheet`` | ``to_excel()`` |
+----------+---------------------------------------+-----------------------------+------------------------------------------------------------------------------------------+
| XLS | ``application/vnd.ms-excel`` | ``to_excel()`` |
+----------+---------------------------------------+-----------------------------+------------------------------------------------------------------------------------------+
| PNG | ``image/png`` | ``plot()`` | Currently not very customizable, but a simple way to view the data as an image. |
+----------+---------------------------------------+-----------------------------+------------------------------------------------------------------------------------------+
| SVG | ``image/svg`` | ``plot()`` | Eventually these could become a fallback for clients that can't handle d3.js |
+----------+---------------------------------------+-----------------------------+------------------------------------------------------------------------------------------+

See the implementation notes below for more details.

Expand All @@ -130,31 +137,38 @@ a single ``TimeSeries`` model.
from rest_pandas import PandasView
from .models import TimeSeries
class TimeSeriesView(PandasView):
# Django REST Framework 2.4
model = TimeSeries

# Django REST Framework 3+
queryset = TimeSeries.objects.all()
serializer_class = TimeSeriesSerializer

# In response to get(), the underlying Django REST Framework ListAPIView
# will load the default queryset (self.model.objects.all()) and then pass
# it to the following function.
# will load the queryset and then pass it to the following function.

def filter_queryset(self, qs):
# At this point, you can filter queryset based on self.request or other
# settings (useful for limiting memory usage)
return qs

# Then, the included PandasSerializer will serialize the queryset into a
# simple list of dicts (using the DRF ModelSerializer). To customize
# which fields to include, subclass PandasSerializer and set the
# appropriate ModelSerializer options. Then, set the serializer_class
# property on the view to your PandasSerializer subclass.
# Then, the default serializer (typically a DRF ModelSerializer) should
# serialize each row in the queryset into a simple dict format. To
# customize which fields to include, create a subclass of ModelSerializer
# and assign it to serializer_class on your view.

# Next, the included PandasSerializer will load the ModelSerializer result
# into a DataFrame and pass it to the following function on the view.

# Next, the PandasSerializer will load the ModelSerializer result into a
# DataFrame and pass it to the following function on the view.

def transform_dataframe(self, dataframe):
# Here you can transform the dataframe based on self.request
# (useful for pivoting or computing statistics)
return dataframe


# For more control over dataframe creation, subclass PandasSerializer and
# set pandas_serializer_class on the view. (Or set list_serializer_class
# on your ModelSerializer subclass' Meta class if you're using DRF 3).

# Finally, the included Renderers will process the dataframe into one of
# the output formats below.

Expand Down
5 changes: 5 additions & 0 deletions rest_pandas/renderers.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
from rest_framework.renderers import BaseRenderer
from rest_framework import status
from tempfile import mkstemp
from pandas import DataFrame

try:
# Python 2 (uses str)
Expand All @@ -24,6 +25,10 @@ def render(self, data, accepted_media_type=None, renderer_context=None):
if not status.is_success(status_code):
return "Error: %s" % data.get('detail', status_code)

if not isinstance(data, DataFrame):
raise Exception(
"Response data is a %s, not a DataFrame!" % type(data)
)
name = getattr(self, 'function', "to_%s" % self.format)
function = getattr(data, name, None)
if not function:
Expand Down
40 changes: 23 additions & 17 deletions rest_pandas/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,27 @@
from pandas import DataFrame


class PandasBaseSerializer(serializers.Serializer):
if hasattr(serializers, 'ListSerializer'):
# Django REST Framework 3
BaseSerializer = serializers.ListSerializer
USE_LIST_SERIALIZERS = True
else:
# Django REST Framework 2
BaseSerializer = serializers.Serializer
USE_LIST_SERIALIZERS = False


class PandasSerializer(BaseSerializer):
"""
Transforms dataset into a dataframe and appies an index
Transforms dataset into a dataframe and applies an index
"""
read_only = True
index_none_value = None

def get_index(self, dataframe):
model_serializer = getattr(self, 'child', self)
if getattr(model_serializer.Meta, 'model', None):
return ['id']
return None

def get_dataframe(self, data):
Expand All @@ -33,30 +46,23 @@ def transform_dataframe(self, dataframe):

@property
def data(self):
data = super(PandasBaseSerializer, self).data
data = super(PandasSerializer, self).data
if data:
dataframe = self.get_dataframe(data)
return self.transform_dataframe(dataframe)
else:
return DataFrame([])


class PandasSimpleSerializer(PandasBaseSerializer):
class SimpleSerializer(serializers.Serializer):
"""
Simple serializer for non-model (simple) views
"""
def get_default_fields(self):
if not self.object:
return {}
return {
name: serializers.Field()
for name in self.object[0].keys()
}

# DRF 3
def to_representation(self, obj):
return obj

class PandasSerializer(PandasBaseSerializer, serializers.ModelSerializer):
"""
Serializer for model views.
"""
def get_index(self, dataframe):
return ['id']
# DRF 2
def to_native(self, obj):
return obj
53 changes: 40 additions & 13 deletions rest_pandas/views.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,9 @@
from django.conf import settings
from rest_framework.settings import perform_import

from .serializers import PandasSimpleSerializer, PandasSerializer
from .serializers import (
SimpleSerializer, PandasSerializer, USE_LIST_SERIALIZERS
)

PANDAS_RENDERERS = getattr(settings, "PANDAS_RENDERERS", None)
if PANDAS_RENDERERS is None:
Expand All @@ -24,36 +26,61 @@
PANDAS_RENDERERS = perform_import(PANDAS_RENDERERS, "PANDAS_RENDERERS")


class PandasSimpleView(APIView):
class PandasMixin(object):
renderer_classes = PANDAS_RENDERERS
paginate_by = None
pandas_serializer_class = PandasSerializer

def with_list_serializer(self, cls):
if not USE_LIST_SERIALIZERS:
# Django REST Framework 2 used the instance serializer for lists
class SerializerWithListSerializer(
self.pandas_serializer_class, cls):
pass
else:

# DRF3 uses a separate list_serializer_class; set if not present
meta = getattr(cls, 'Meta', object)
if getattr(meta, 'list_serializer_class', None):
return cls

class SerializerWithListSerializer(cls):
class Meta(meta):
list_serializer_class = self.pandas_serializer_class

return SerializerWithListSerializer


class PandasSimpleView(PandasMixin, APIView):
"""
Simple (non-model) Pandas API view; override get_data
with a function that returns a list of dicts.
"""
serializer_class = PandasSimpleSerializer
renderer_classes = PANDAS_RENDERERS
serializer_class = SimpleSerializer

def get_data(self, request, *args, **kwargs):
return []

def get(self, request, *args, **kwargs):
data = self.get_data(request, *args, **kwargs)
serializer = self.serializer_class(data, many=True)
serializer_class = self.with_list_serializer(self.serializer_class)
serializer = serializer_class(data, many=True)
return Response(serializer.data)


class PandasView(ListAPIView):
class PandasView(PandasMixin, ListAPIView):
"""
Pandas-capable model list view
"""
model_serializer_class = PandasSerializer
renderer_classes = PANDAS_RENDERERS
paginate_by = None
def get_serializer_class(self, *args, **kwargs):
cls = super(PandasView, self).get_serializer_class(*args, **kwargs)
return self.with_list_serializer(cls)


class PandasViewSet(ListModelMixin, GenericViewSet):
class PandasViewSet(PandasMixin, ListModelMixin, GenericViewSet):
"""
Pandas-capable model ViewSet (list only)
"""
model_serializer_class = PandasSerializer
renderer_classes = PANDAS_RENDERERS
paginate_by = None
def get_serializer_class(self, *args, **kwargs):
cls = super(PandasViewSet, self).get_serializer_class(*args, **kwargs)
return self.with_list_serializer(cls)
7 changes: 7 additions & 0 deletions tests/testapp/serializers.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
from rest_framework.serializers import ModelSerializer
from .models import TimeSeries


class TimeSeriesSerializer(ModelSerializer):
class Meta:
model = TimeSeries
11 changes: 9 additions & 2 deletions tests/testapp/views.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
from rest_pandas import PandasSimpleView, PandasView, PandasViewSet
from .models import TimeSeries
from .serializers import TimeSeriesSerializer


class NoModelView(PandasSimpleView):
Expand All @@ -13,8 +14,14 @@ def get_data(self, request, *args, **kwargs):


class TimeSeriesView(PandasView):
model = TimeSeries
queryset = TimeSeries.objects.all()
serializer_class = TimeSeriesSerializer

def transform_dataframe(self, df):
df['date'] = df['date'].astype('datetime64[D]')
return df


class TimeSeriesViewSet(PandasViewSet):
model = TimeSeries
queryset = TimeSeries.objects.all()
serializer_class = TimeSeriesSerializer