Working on support for stream resources

[thebigb]

I'm working on a project that has a component that needs a bit of performance, so I figured PHP-CPP could do there. However, the component works on a large amount of data, and I was sad to find PHP-CPP doesn't support stream resources as value yet. I could go around passing temporary files around, but that's not a very pretty solution.

So I decided to get my hands dirty and created a working prototype: thebigb@ad87eeb

Currently on my TODO list:

• Write tests
• Perhaps standardize the Stream class to something like std::iostream?
• Fix race condition(?) - the stream doesn't always load from the zval when a non-existing file is opened in PHP code with a+ and used immediately in an extension

Would you guys be interested in merging a more mature version of this feature? I'd be happy to accept suggestions.

[EmielBruijntjes]

Yes I am interested. I like your coding style, it matches the PHP-CPP coding.

I would be nice if the interface could be so that in your C++ code you can get a stream variable passed in from user space, and turn that into a full blown C++ std::iostream object:

void my_function(Php::Parameters &params)
{
    // access to the one and only parameter
    Php::Value param = params[0];

    // it must be a stream (@todo report error)
    if (!param.isStream()) return;

    // wrap in a stream object, so that we can treat it as C++ stream
    Php::Stream stream(param);

    // and now it can be used as if it is a normal std::iostream
    stream << "input" << std::endl;
}

Would be nice if it would work the other way around too. From your C++ code you can create a C++ stream, and pass that on to PHP user space where it will continue to live as PHP stream:

Php::Value my_function()
{
    // return a PHP stream resource, in which a C++ stream is wrapped
    return Php::Stream(std::cout);
}

This is just a first idea, don't known if this is possible.

[thebigb]

Thanks for the feedback!

I noticed that I hadn't included a usage example of the prototype. So here's the test code I've been using:

$fp = fopen('test.txt', 'w+');
writeHelloWorld($fp);

void writeHelloWorld(Php::Parameters &params)
{
    // If the Php::Value is not a stream, an exception will be thrown during the implicit cast
    Php::Stream *stream = params[0];
    stream->write("Hello World!");
    stream->flush();
}

I'm not sure yet how much of the std::iostream interface I will be able to implement, because the php_stream interface is somewhat limited compared to the std::iostream interface. Some function can be emulated, but non-seekable streams like sockets will be challenging. Then again, I could also throw a NotSupportedException of some sorts.

The C++ stream to userland is a great idea; I hadn't considered that yet. It might be possible by backing a StreamWrapper with a std::iostream, but I could also do some hacking with internal php_stream struct, since it doesn't seem too complicated.

I'll post any updates here. I'd appreciate if someone could review my changes once in a while.

[EmielBruijntjes]

How are you doing? Based on your initial work, I've also been experimenting with streams today, I've created a branch with the things that I've added so far. See https://github.com/CopernicaMarketingSoftware/PHP-CPP/tree/streams

You can use the Php::Stream class to write C++ functions that accept streams as parameters. This is basically the same as your implementation, but in my setup I wrap a Php::Stream object around a value instead of calling the value.stream() method:

void my_function(Php::Parameters &params)
{
    Php::Stream stream(params[0]);
    stream.write("some random text");
}

However, I do not know if this setup is the most logic one. It already is possible (even today) to work with streams by using normal Php functions. The above function could also be implemented like this:

void my_function(Php::Parameters &params)
{
    Php::Value fp = params[0];
    Php::fwrite(fp, "some random text");
}

Maybe this second solution is even more readable for PHP programmers, because they already know functions like Php::fwrite(), and they do not have to learn the details of a new Php::Stream class.

With the Php::Stream class, you can also make C++ functions that create/open new streams:

Php::Value my_open_file()
{
    return Php::Stream("/tmp/my-output-file.txt","w");
}

But once again, a similar thing would already be possible if you use the Php::fopen() function:

Php::Value my_open_file()
{
    return Php::fopen("/tmp/my-output-file.txt","w");
}

So, it is very debatable if we really need such a special Php::Stream class, as you can already use the old-fashioned PHP functions to achieve exactly the same, without the need for a special class.

The second thing that I was looking at was a way to create custom streams. Imagine that you want to create a C++ function that creates a custom PHP stream, for example a stream that discards data:

class DevNullStream : public StreamImpl
{
public:
    virtual size_t write(const char *data, size_t size) override
    {
        // all data was written
        return size;
    }

    // @todo add a simple implementation for the other virtual 
    // functions from the StreamImpl base class

};

Php::Value open_dev_null()
{
    return new DevNullStream();
}

As i said, this all is still very experimental because I have not decided if we really need this, and if we do, what the best thinkable API would be. And besides that, the underlying "streams" implementation of PHP is not so structured. I do not really like it, and it is a challenge to write a C++ layer on top of it, that completely hide the internals of Zend, and offers a nice API instead.

And a second thing to consider is this. PHP streams are old concepts from the very early days of the PHP language, and these streams in turn are based on the concept of filedescriptors that goes all the way back to the 1970's or so. In an object oriented world, filedescriptors and streams seem (to me) a little outdated. Instead of writing functions like "fopen()" and "fsockopen()" to open a file or socket stream, it makes more sense to create classes like "File" and "Socket" that do the same instead. So if you're tempted to write an extension that opens/creates new streams, it could very well be a better idea to create a class for that.

[thebigb]

Hi Emiel,

I've been swamped with work lately. However, I have done some research on the iostream part. I've come to the conclusion that I should either extend std::streambuf or use Boost.Iostreams. I'll see if I can find some time this week to get a prototype or two running.

See https://github.com/CopernicaMarketingSoftware/PHP-CPP/tree/streams
in my setup I wrap a Php::Stream object around a value instead of calling the value.stream() method

Neat, that looks much more elegant.

So, it is very debatable if we really need such a special Php::Stream class, as you can already use the old-fashioned PHP functions to achieve exactly the same, without the need for a special class.

Perhaps I'm missing the point, but why not have both?

The second thing that I was looking at was a way to create custom streams.

The way I originally thought of doing that is to just accept any std::iostream implementation, and then through some wizardry point the _php_stream_ops function pointers to the std::iostream methods. I could be underestimating the complexity of this, though.

In an object oriented world, filedescriptors and streams seem (to me) a little outdated. Instead of writing functions like "fopen()" and "fsockopen()" to open a file or socket stream, it makes more sense to create classes like "File" and "Socket" that do the same instead. So if you're tempted to write an extension that opens/creates new streams, it could very well be a better idea to create a class for that.

Could you elaborate?

[EmielBruijntjes]

About using the Boost libraries: please don't do that. I do not want to add that dependency. PHP-CPP should compile and work on platforms without Boost. The only dependency should be the one on std.

About using both a Php::Stream class and regular functions. Of course we can have both, my question is if we need both, and if we do actually make life better by having two implementations. The simplest solution is to stick to exactly the same API as PHP user space functions: they are already documented, tested, and people know how they work. Simple solutions are often the best. A new Php::Stream class may introduce all sort of problems.

My problem with streams vs object orientation is that the same abstract concept (a stream) is used for things that are completely different: a listening UDP socket, a directory opened for reading, and a file opened for writing are all stream resources, but they have almost nothing in common! In my opinion, PHP already has gone too far by pretending that all these things are essentially similar. They are not. With object orientation you can do this much more elegant - a socket and file could both have the same base class to support reading and writing, but have different methods when it comes down to finding out the IP address of the socket, or to seek within the file.

On top of that, the stream implementation inside the Zend engine is not something to be proud of, and to encapsulate that mess in a C++ class, and have a logical API at the same time -- that is not going to be easy.

And if we do manage to add a simple 'create-your-own-stream' feature to PHP-CPP however, it is just as if we approve streams, and motivate users to make even more stream-like-things - while I am in reality not so in favor of them.

All these critical remarks do not necessary mean that I am completely against it. The nice thing about creating a Php::Stream class is the intellectual challenge to make a logical and easy API that allows people to make custom streams - but once it is ready we must hope that people will refrain from using it, and use object orientation instead.

[marketdev8]

Thanks for this guys, very useful, and I hope you add it to the main branch.

[AndrewCarterUK]

Hi all,

I'm currently writing an extension that has to use stream resources. At the moment I have to call the PHP functions to create and manipulate these resources - which isn't ideal.

I love the library, it's made writing an extension really easy - I just thought I would mention here that I think this Php::Stream feature would be really welcome!

Also - with PSR-7 stream resources are going to become a lot more common in PHP code!

Thanks,

Andrew

[kolesar-andras]

Based on work of Bernardo created a branch that makes Php::Stream available on PHP 7.