Skip to content

Commit

Permalink
update signature of ReadFile() to match underlying API
Browse files Browse the repository at this point in the history
This commit updates the signature of ReadFile() to
better match the underlying Windows API. By doing so
we should be able to better support overlapped I/O and
give the user of this function greater control over the
input and output.

For a complete discussion around why this change was made
see #141.
  • Loading branch information
opalmer committed Dec 10, 2017
1 parent 86ed6d8 commit 5434743
Show file tree
Hide file tree
Showing 4 changed files with 69 additions and 37 deletions.
4 changes: 2 additions & 2 deletions docs/source/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ Notable enhancements and changes are:

* :issue:`141` - :func:`pywincffi.kernel32.file.ReadFile` has been updated
to provide improved support for overlapped IO. This is a **breaking change**
because the type of the returned value has changed and requires additional
processing.
because the return value and signature of the ReadFile function has
changed.

0.5.0
~~~~~
Expand Down
50 changes: 33 additions & 17 deletions pywincffi/kernel32/file.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

from pywincffi.core import dist
from pywincffi.core.checks import NON_ZERO, input_check, error_check, NoneType
from pywincffi.exceptions import WindowsAPIError
from pywincffi.exceptions import WindowsAPIError, InputError
from pywincffi.wintypes import (
SECURITY_ATTRIBUTES, OVERLAPPED, HANDLE, wintype_to_cdata
)
Expand Down Expand Up @@ -193,9 +193,10 @@ def FlushFileBuffers(hFile):
error_check("FlushFileBuffers", code=code, expected=NON_ZERO)


def ReadFile(hFile, nNumberOfBytesToRead, lpOverlapped=None):
# TODO: Currently lpBuffer only supports bytearray. Other types could be?
def ReadFile(hFile, lpBuffer, nNumberOfBytesToRead, lpOverlapped=None):
"""
Read the specified number of bytes from ``hFile``.
Reads data from the specified file or I/O device.
.. seealso::
Expand All @@ -204,10 +205,18 @@ def ReadFile(hFile, nNumberOfBytesToRead, lpOverlapped=None):
:param pywincffi.wintypes.HANDLE hFile:
The handle to read from.
:param bytearray lpBuffer:
The array to receive the data that was read.
.. note::
The provided ``lpBuffer`` must be at least as large
as ``nNumberOfBytesToRead``.
:param int nNumberOfBytesToRead:
The number of bytes to read from ``hFile``
The maximum number of bytes to be read.
:keyword pywincffi.wintypes.OVERLAPPED lpOverlapped:
:keyword OVERLAPPED lpOverlapped:
See Microsoft's documentation for intended usage and below for
an example.
Expand All @@ -220,31 +229,38 @@ def ReadFile(hFile, nNumberOfBytesToRead, lpOverlapped=None):
>>> read_data = ReadFile( # read 12 bytes from hFile
... hFile, 12, lpOverlapped=lpOverlapped)

This comment has been minimized.

Copy link
@exvito

exvito Dec 12, 2017

Contributor

This example needs to be updated.

This comment has been minimized.

Copy link
@opalmer

opalmer Jan 1, 2018

Author Owner

Fixed in f90b4ed.

:raises InputError:
In addition to being raised for type issues :class:`InputError` will
also be raised if the size of the provided ``lpBuffer`` is smaller
than ``nNumberOfBytesToRead``.
:returns:
Returns a Python bytearray. If the input ``hFile`` was not
opened in overlapped mode then the returned array will contain
the resulting data. If ``hFile`` was opened in overlapped mode
then the read data will be pushed into returned array when the
read has completed.
This function returns the number of bytes read by the underlying
ReadFile() function as an interger.
"""
ffi, library = dist.load()

input_check("hFile", hFile, HANDLE)
input_check("lpBuffer", lpBuffer, bytearray)
input_check("nNumberOfBytesToRead", nNumberOfBytesToRead, integer_types)
input_check(
"lpOverlapped", lpOverlapped,
allowed_types=(NoneType, OVERLAPPED)
)

readBuffer = bytearray(nNumberOfBytesToRead)
lpBuffer = ffi.from_buffer(readBuffer)
if len(lpBuffer) < nNumberOfBytesToRead:
raise InputError(
"ReadFile", None,
message="The length of lpBuffer is {} which is smaller than "
"`nNumberOfBytesToRead` ({}).".format(

This comment has been minimized.

Copy link
@exvito

exvito Dec 12, 2017

Contributor

Nitpicking for consistency: either use quoted lpBuffer to match the quoted nNumberOfBytesToRead or unquote the latter.

This comment has been minimized.

Copy link
@opalmer

opalmer Jan 1, 2018

Author Owner

Fixed in 985bcda.

len(lpBuffer), nNumberOfBytesToRead))

ffi, library = dist.load()
bytes_read = ffi.new("LPDWORD")
code = library.ReadFile(
wintype_to_cdata(hFile), lpBuffer, nNumberOfBytesToRead, bytes_read,
wintype_to_cdata(lpOverlapped)
wintype_to_cdata(hFile), ffi.from_buffer(lpBuffer),
nNumberOfBytesToRead, bytes_read, wintype_to_cdata(lpOverlapped)
)
error_check("ReadFile", code=code, expected=NON_ZERO)

This comment has been minimized.

Copy link
@exvito

exvito Dec 12, 2017

Contributor

From the Microsoft docs, it can return 0 when completing asynchronously. Suggest expected = NON_ZERO if lpOverlapped is None else 0 as in WriteFile at around line 173.

In that case, we may want to go a check GetLastError confirming we get ERROR_IO_PENDING (more details in the docs). BTW: looking at my implementation of WriteFile, we may need to bring this extra check there, too.

This comment has been minimized.

Copy link
@opalmer

opalmer Jan 1, 2018

Author Owner

Fixed in 1116004 for ReadFile. WriteFile already has this check fixed.

return readBuffer
return bytes_read[0]


def MoveFileEx(lpExistingFileName, lpNewFileName, dwFlags=None):
Expand Down
35 changes: 23 additions & 12 deletions tests/test_kernel32/test_file.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,7 @@

from pywincffi.core import dist
from pywincffi.dev.testutil import TestCase
from pywincffi.exceptions import WindowsAPIError

from pywincffi.exceptions import WindowsAPIError, InputError
from pywincffi.kernel32 import file as _file # used for mocks
from pywincffi.kernel32 import (
CreateFile, CloseHandle, MoveFileEx, WriteFile, FlushFileBuffers,
Expand Down Expand Up @@ -69,21 +68,33 @@ def _handle_to_read_file(self, path):
self.addCleanup(CloseHandle, hFile)
return hFile

def test_write_then_read_bytes_ascii(self):
path, written = self._create_file(b"test_write_then_read_bytes_ascii")
def test_lpBuffer_smaller_than_nNumberOfBytesToRead(self):
path, _ = self._create_file(b"")
hFile = self._handle_to_read_file(path)
self.assertEqual(
ReadFile(hFile, written), b"test_write_then_read_bytes_ascii")

def test_write_then_read_null_bytes(self):
path, written = self._create_file(b"hello\x00world")
with self.assertRaisesRegex(
InputError, r".*The length of lpBuffer is.*"
):
ReadFile(hFile, bytearray(0), 1)

def test_lpBuffer_equal_to_nNumberOfBytesToRead(self):
path, _ = self._create_file(b"")
hFile = self._handle_to_read_file(path)
self.assertEqual(ReadFile(hFile, written), b"hello\x00world")
ReadFile(hFile, bytearray(1), 1) # Should not raise exception

def test_write_then_read_partial(self):
path, _ = self._create_file(b"test_write_then_read_partial")
def test_read(self):
path, written = self._create_file(b"hello world")
self.assertEqual(written, 11)
hFile = self._handle_to_read_file(path)
self.assertEqual(ReadFile(hFile, 4), b"test")
lpBuffer = bytearray(written)
read = ReadFile(hFile, lpBuffer, 5)
self.assertEqual(read, 5)

This comment has been minimized.

Copy link
@exvito

exvito Dec 12, 2017

Contributor

Recall, the number of read bytes may be less than the requested 5. Recommend adjusting this assertion,

self.assertEqual(lpBuffer[:read], bytearray(b"hello"))

This comment has been minimized.

Copy link
@exvito

exvito Dec 12, 2017

Contributor

Recall, the number of read bytes may be less than the requested 5. Recommend adjusting this assertion.


# The rest of the buffer is essentially unmanaged but let's be sure
# the remainder of the buffer has not been modified by ReadFile().
self.assertEqual(
lpBuffer[read:], bytearray(b"\x00\x00\x00\x00\x00\x00"))

This comment has been minimized.

Copy link
@exvito

exvito Dec 12, 2017

Contributor

Recall, the number of read bytes may be less than the requested 5. Recommend adjusting this assertion.



class TestMoveFileEx(TestCase):
Expand Down
17 changes: 11 additions & 6 deletions tests/test_kernel32/test_pipe.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,9 +55,10 @@ def test_python_bytes(self):

data = b"hello world"
bytes_written = WriteFile(writer, data)
self.assertEqual(
ReadFile(reader, bytes_written),
b"hello world")
buffer = bytearray(bytes_written)
read = ReadFile(reader, buffer, bytes_written)
self.assertEqual(bytes_written, read)

This comment has been minimized.

Copy link
@exvito

exvito Dec 12, 2017

Contributor

Recall, the number of read bytes may be less than the requested bytes_written. Recommend adjusting this assertion.

self.assertEqual(buffer, bytearray(b"hello world"))

This comment has been minimized.

Copy link
@exvito

exvito Dec 12, 2017

Contributor

Recall, the number of read bytes may be less than the requested. Recommend adjusting this assertion.


_, library = dist.load()
self.maybe_assert_last_error(library.ERROR_INVALID_HANDLE)
Expand All @@ -79,9 +80,11 @@ def test_peek_does_not_remove_data(self):

data = b"hello world"
data_written = WriteFile(writer, data)

buf = bytearray(data_written)
read = ReadFile(reader, buf, data_written)
self.assertEqual(read, data_written)

This comment has been minimized.

Copy link
@exvito

exvito Dec 12, 2017

Contributor

Recall, the number of read bytes may be less than the requested. Recommend adjusting this assertion.

PeekNamedPipe(reader, 0)
self.assertEqual(ReadFile(reader, data_written), data)
self.assertEqual(buf, bytearray(data))

This comment has been minimized.

Copy link
@exvito

exvito Dec 12, 2017

Contributor

Recall, the number of read bytes may be less than the requested. Recommend adjusting this assertion.

_, library = dist.load()
self.maybe_assert_last_error(library.ERROR_INVALID_HANDLE)

Expand Down Expand Up @@ -125,7 +128,9 @@ def test_total_bytes_avail_after_read(self):
bytes_written = WriteFile(writer, data)

read_bytes = 7
ReadFile(reader, read_bytes)
buf = bytearray(read_bytes)
read = ReadFile(reader, buf, read_bytes)
self.assertEqual(read, read_bytes)

This comment has been minimized.

Copy link
@exvito

exvito Dec 12, 2017

Contributor

Recall, the number of read bytes may be less than the requested. Recommend adjusting this assertion.


result = PeekNamedPipe(reader, 0)
self.assertEqual(
Expand Down

0 comments on commit 5434743

Please sign in to comment.