Upload big files to s3 by URL

Recently I needed to upload files to s3 having URL. Unfortunately I didn’t find a way how can I do that with native AWS php sdk. So I needed to provide either file or file body Upload an Object Using the AWS SDK for PHP . I didn’t want to care about temp files. So the easiest solution that came up to my head were

// @see https://github.com/2amigos/resource-manager/
$body = @get_file_content($url);
Yii::app()->resourceManager->save($body, $path, $options);

That worked fine, I also could get file size and mime type having file body. Maybe not very elegant but it worked. Until…

    /**
     * Get file size from file content
     * @param $string
     * @return int
     */
    private function fileSizeFromString($string)
    {
        if (function_exists('mb_strlen')) {
            $size = mb_strlen($string, '8bit');
        } else {
            $size = strlen($string);
        }
        return $size;
    }

    /**
     * Return image mime type
     * @param $string
     * @return null
     */
    private function mimeFromString($string)
    {
        $finfo = new finfo(FILEINFO_MIME);
        $mime = $finfo->buffer($string);
        if (!empty($mime)) {
            $mime = explode(';', $mime);
            return $mime[0];
        }
        return null;
    }

And everything were fine until I tried to do that with big files. Also another problem that occurred – I did this script from console and it just stopped working without any message. Yeah it was fatal error with memory and yes I didn’t know about that because had @file_get_contents in order not to have errors with dead URLs. But that’s another story.

Another simple solution that came to my head – save file. But that’s is not what I wanted at all – I didn’t want to take care about files. Then I found out that s3 allows to work with \Guzzle\Http\EntityBodyInterface.

Guzzle has stream and they are really cool http://docs.guzzlephp.org/en/guzzle4/streams.html  based on php streams functions http://php.net/manual/en/intro.stream.php they allow you not to care about files.

So what do we finally get.

    protected function getTempFileFromUrl($url, &$body, &$mime, &$size)
    {
        $client = new Client([
            'debug' => false,
            //'connect_timeout' => 180,
            'headers' => [
                'Accept' => '* // *'
            ]
        ]);

        try {
            $response = $client->request('GET', $url, [
                'allow_redirects' => true
            ]);

            if ($response->getStatusCode() == 200) {
                $mime = $response->getHeader('content-type');
                $mime = $mime ? $mime[0] : null;
                $size = $response->getHeader('content-length');
                $size = $size ? $size[0] : null;
                $body = $response->getBody();
                return true;
            } else {
                return false;
            }
        } catch (\Exception $e) {
            Yii::log("Cannot get file by URL. Details:" . $e->getMessage() .  "\n" . $e->getTraceAsString());
            return false;
        }
    }

    /**
      * This is common models that save file info - s3 path, mime, size etc.
      */ 
    public function createFromUrl($url, $path, $type)
    {
        if (!$this->getTempFileFromUrl($url, $body, $mime, $size)) {
            $this->addError('path', 'Cannot get file from URL');
            return false;
        }

        $this->path = $path;
        $this->size = $this->$size;
        $this->mime = $mime;
        $this->type = $type;
        $this->created_by = $this->getCreatedById();
        if (!$this->validate()) {
            return false;
        }

        $options = [
            'ContentType' => $mime,
        ];
        if (Yii::app()->resourceManager->save($body, $path, $options)) {
            return $this->save(false);
        } else {
            $this->addError('path', 'Cannot save image to S3');
        }
        return false;
    }

Leave a Reply

Your email address will not be published. Required fields are marked *