tag

Monday, 06 June 2022

3 months ago

7 minutes read

  • Laravel
  • GraphQL
  • Testing
  • Backend
  • PHP
  • Minio
  • S3
  • Buckets

Laragraph - S3 my thumbnails - Part 3

Are we seriously building a blog API without the possibility to upload a thumbnail?! Nah! We have to solve that! For this third part of Laragraph we will see how can we upload files to S3 with Laravel and GraphQL. Do you like this topic? Stick around and come with me during this journey of learning things 😎

Tiago Sousa

Tiago Sousa

Software Engineer

Introductionlink

I don't know about you, but for me uploading files is always the trickiest and funniest part when building a product. How should I save the file? How should I validate the file? Where should I save the file? The feedback that I got from the poll that I created on twitter shows that most of you probably feel the same way as I do so let's have a look into this topic!

Before cloud native was a thing, the files were being saved in the same place where the code for the application was deployed, normally in a VPS (Virtual Private Server). As you can imagine this may bring about some problems... What if your VPS dies? You lose all the images? Having this example in mind and many others that software engineers were facing a solution called S3 came into the market. S3 stands for Simple Storage Service and it's basically an object storage service that offers scalability, data availability, security, and performance. You know... all those things that cloud native services and applications offer!

Nowadays, almost every single cloud provider has a S3 service like Amazon S3, Google Cloud Storage or even Azure Storage.

This solution really changed the way that engineers develop their applications and with so much traction some on perm solutions like MinIO appeared.

MinIOlink

MinIO is the perfect solution for your development environment as you can simply define it in your docker-compose and use it out of the box, instead of subscribing and paying for object storage for your team's development environment. If you want to manage your own S3 service you can deploy MinIO in multipe ways and host it in your cloud native environment, independently of the cloud provider you are using. But today we are not talking about cloud in depth, so let's focus on our problem and start configuring MinIO for our development environment.

Laravel Sail already comes with support for MinIO, which is great! It will take a little bit of boilerplate and configuration from our side. To install MinIO let's just re-run the install command and select both MySQL and MinIO by typing 0,6 after running the following command:

sail artisan sail:install
copy

NOTE:

If you have php installed on your machine you can also use php artisan sail:install

Sail install command

NOTE:

You need to have your containers running before running the sail:install command. For that run sail up -d.

After this, your docker-compose file should be updated and MinIO included and configured! Before checking if everything is working with our development environment we still need to update our .env file to tell laravel to use s3 as our drive for the file system:

.env

FILESYSTEM_DISK=s3
copy

We also need to update our environment variables regaring S3 configuration

.env

AWS_ACCESS_KEY_ID=sail
AWS_SECRET_ACCESS_KEY=password
AWS_DEFAULT_REGION=us-east-1
AWS_BUCKET=laragraph
AWS_ENDPOINT=http://minio:9000
AWS_USE_PATH_STYLE_ENDPOINT=true
copy

NOTE:

Laravel uses AWS_* environment variables to configure the file system but it will work for every cloud provider, you just need to put the correct values.

Now that we have everything installed and configured let's check if everything is okay by starting our docker development environment:

sail up -d
copy

If we did everything correctly, our MinIO should be available at http://localhost:8900/ MinIO Login Screen

NOTE:

The username and password should be the values defined for the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

When we changed our application environment variables we added AWS_BUCKET=laragraph, but when starting the MinIO container it doesn't come with any default bucket... For that we can create a docker container that will use MinIO client (which is called mc) in order to create a default bucket for us called laragraph as we previously configured in our .env.

docker-compose.yaml

minio-buckets:
image: minio/mc:latest
environment:
AWS_ACCESS_KEY_ID: '${AWS_ACCESS_KEY_ID}'
AWS_SECRET_ACCESS_KEY: '${AWS_SECRET_ACCESS_KEY}'
AWS_ENDPOINT: '${AWS_ENDPOINT}'
depends_on:
- minio
networks:
- sail
entrypoint: >
/bin/sh -c "
/usr/bin/mc alias set laragraph $AWS_ENDPOINT $AWS_ACCESS_KEY_ID $AWS_SECRET_ACCESS_KEY;
/usr/bin/mc mb laragraph/laragraph;
/usr/bin/mc policy set download laragraph/laragraph;
"
copy

Let's run our development environment again in order to start the container that will create the default bucket for laragraph.

sail up -d
copy

If you refresh your MinIO you should now have a default bucket called laragraph 😎

MinIO Laragraph Bucket

Testing the uploadlink

At this point everything should be configured, so let's see if we can get images uploaded to MinIO from our laravel application. I often define a /foo route in my web.php file to test these kind of things! For this test let's just try to drag a random image to your bucket and access it with laravel Storage facade. If we are able to, then we are sure that everything is working as intended!

routes/web.php

use Illuminate\Support\Facades\Storage;
Route::get('/foo', function () {
return Storage::allFiles();
});
copy

Now if you access the /foo endpoint you should see an array of images that you dragged into our laragraph bucket, right? Well, theoretically yes, but unfortunately no ... At this point we are presented with an error saying:

Class "League\Flysystem\AwsS3V3\PortableVisibilityConverter" not found

Which basically means that we are missing a dependency so let's fix that!

sail composer require league/flysystem-aws-s3-v3
copy

Now, let's try again and refresh the /foo endpoint πŸ‘€ Yeahh, now we're talking! An array of images from our MinOI instance! πŸ₯³ With this we are certain that everything is well configured and we are ready to write some code πŸ”₯ (I know that you were waiting for this πŸ˜‰)

Take me to the cloudslink

Lighthouse facilitates the upload of files by providing a scalar for it as you can see here.

In order to enable this scalar, we have to define it in our schema. I like to have all scalars defined at the beginning of the file, but you can add them anywhere.

graphql/schema.graphql

"Can be used as an argument to upload files using https://github.com/jaydenseric/graphql-multipart-request-spec"
scalar Upload @scalar(class: "Nuwave\\Lighthouse\\Schema\\Types\\Scalars\\Upload")
copy

After adding the scalar, we need to use it! In our Article type let's add a new field: The thumbnail

graphql/articles.graphql

type Article {
"Unique primary key." id: ID!
"Unique slug address." slug: String!
"Non-unique title." title: String!
"Non-unique body." body: String!
+ "Thumbnail path" + thumbnail: String
"The guardor of the article" user: User! @belongsTo
"When the article was created." created_at: DateTime!
"When the article was last updated." updated_at: DateTime! }
copy

Well, it looks like we have a new field! So we need to:

  • update our CreateArticleInputValidator in the schema
  • update our migrations
  • update our models
  • update our validators
  • update our logic to upload the thumbnail

Starting with the schema let's add the scalar that we previously defined

graphql/articles.graphql

input CreateArticleInput @validator {
slug: String!
title: String!
body: String!
thumbnail: Upload
}
copy

Now let's add the thumbnail field as a nullable field.

database/migrations/create_articles_table.php

public function up()
{
Schema::create('articles', function (Blueprint $table) {
$table->id();
$table->string('slug')->unique();
$table->string('title');
$table->text('body');
$table->string('thumbnail')->nullable();
$table->timestamps();
$table->foreignIdFor(User::class)
->constrained()
->cascadeOnDelete()
->cascadeOnUpdate();
});
}
copy

The third step is to update our model and add the thumbnail as a fillable field.

app/Models/Article.php

protected $fillable = [
'title',
'body',
'slug',
'thumbnail',
];
copy

Now let's validate our thumbnail. But first let's create a use case just for the sake of this demo project!

Sometimes our user will upload a thumbnail but when he doesn't we want to return a default thumbnail for him! That's why we added the thumbnail field as nullable Regarding validation it's pretty straight forward! Sometimes we want to validate the image, sometimes we don't! Let's see how can we do this in our validation class

app/GraphQL/Validators/CreateArticleInputValidator.php

public function rules(): array
{
return [
'slug' => ['required', Rule::unique('articles', 'slug')],
'title' => ['required'],
'body' => ['required'],
'thumbnail' => ['sometimes', 'image'],
];
}
copy

NOTE:

The image validation comes from laravel and it will ensure that the image has a jpg, jpeg, png, bmp, gif, svg, or webp extension. If you want to have a deeper look into the validation rules feel free to check them here.

To be honest, this can't get more easy, right? Right? .... Well how can we return a default value when the thumbnail is null? Laravel has something called Accessors & Mutators which are ways of manipulating values when they are accessed or set. What we want to do here is to set a default value when the thumbnail is null and for that let's define an Accessors in our Article model.

app/Models/Article.php

/** * The URL for the default thumbnail. * * @var string */
public const DEFAULT_THUMBNAIL_PATH = 'https://placeimg.com/640/380/tech';
/** * Interact with the article thumbnail * * @param string $value * @return \Illuminate\Database\Eloquent\Casts\Attribute */
protected function thumbnail(): Attribute
{
return Attribute::make(
get: fn ($value) => $value ? Storage::url($value) : self::DEFAULT_THUMBNAIL_PATH
);
}
copy

At this point, the only missing step is to change the logic for the store method in the ArticleMutation to allow file uploads!

app/GraphQL/Mutations/ArticleMutation.php

/** * @param null $_ * @param array<string, mixed> $request */
public function store($_, array $request)
{
$user = request()->user();
/** @var \Illuminate\Http\UploadedFile $file */
$file = Arr::has($request, 'thumbnail') ? Storage::putFile("{$user->id}/blog", $request['thumbnail']) : null;
return $user->articles()->create([
...$request,
'thumbnail' => $file
]);
}
copy

You may be thinking that this isn't the correct way to do things and you are right... There are a lot of things that we should ensure here. For example we should only create the article if the image is successfully uploaded or, if the image is uploaded but the article failed it's creation, we should remove the image. I could cover all these topics, but the article would be too extensive so I will let that part open for you to try and implement as well πŸ˜‰

NOTE:

When you are building your product be sure to always have this in mind!

Let me try! πŸ₯Ήlink

Now that we have everything we need implemented for our feature let's try it out! Unfortunately, the graphql playground that we have out of the box available at http://localhost/graphql-playground doesn't provide a way to upload files so we will have to switch to something like postman in order to upload a file or use a curl request.

cURL me!link

In order to understand that both the creation for our articles work with and without the file upload let's make two requests

  • without file upload
curl --location --request POST 'http://localhost/graphql' \
--header 'Authorization: Bearer o1jgkgpywh5Xwt6bUMMXYNvCy3zDGBGFFuj8w8qG' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"mutation {createArticle(input: {slug: \"article-without-thumbnail\" title: \"Article without thumbnail\" body: \"My awesome content 😎\"}) { title slug thumbnail }}"}'
copy

If everything is working as expected for this request we should expect the thumbnail to come with the default thumbnail path that we previously defined and guess what πŸ‘€

{
"data":{
"createArticle":{
"title":"Article without thumbnail",
"slug":"article-without-thumbnail",
"thumbnail":"https://placeimg.com/640/380/tech"
}
}
}
copy
  • with file upload
curl --location --request POST 'http://localhost/graphql' \
--header 'Authorization: Bearer o1jgkgpywh5Xwt6bUMMXYNvCy3zDGBGFFuj8w8qG' \
--form 'operations="{ \"query\": \"mutation CreateArticle($file: Upload!) {createArticle(input: {slug: \\\"article-with-thumbnail\\\" title: \\\"Article with thumbnail\\\" body: \\\"My awesome content 😎\\\" thumbnail: $file}) {title slug thumbnail}}\" }"' \
--form 'map="{ \"0\": [\"variables.file\"] }"' \
--form '[email protected]"/Users/tsousa/Desktop/random-image.png"'
copy

NOTE:

Don't forget to change the last --form value to a valid image path from your machine.

Now if everything is working as expected (which i'm pretty sure it is 😎) the output should be the url for our image!

{
"data":{
"createArticle":{
"title":"Article with thumbnail",
"slug":"article-with-thumbnail",
"thumbnail":"http://minio:9000/laragraph/1/blog/elnoEFL0qCgtaRzhkTXCgMKBAIrJIpIknbqMfYuA.jpg"
}
}
}
copy

POST me MANlink

As I said, we can also use postman to test this out 😎 When configuring the request, don't forget to add your access token in the Authorization tab, to define the request as a POST and the mapping for the file to be type of file.

In order to be easier for you I will add the configuration for the operations and map values below

Operations

{ "query": "mutation CreateArticle($file: Upload!) {createArticle(input: {slug: \"article-with-thumbnail-from-postman\" title: \"Article with thumbnail from postman\" body: \"My awesome content 😎\" thumbnail: $file}) {title slug thumbnail}}" }
copy

Variables

{ "0": ["variables.file"] }
copy

Postman file upload

Don't you dare fail! 😑link

As part of this series I also want to cover testing as I think thats a very important topic and isn't as much time consuming as people think. Let's start by rename the can create an article test to can create an article without thumbnail as the logic changed a little bit. Laravel offers a really nice fake() method as part as the Storage facade that will allow you to easily generate a fake disk that simplifies the testing of file uploads. This method from the Storage facade combined with fake() method from the UploadedFile will make you feel like you are cheating when you are writing tests for file uploads. The fake() method from the UploadedFile will return a File instance that you can use to simulate a real file for testing purposes.

Now that the basics are explained let's replace some code, shall we?

tests/Feature/ArticleTest.php

- it('can create an article', function () { + it('can create an article without thumbnail', function () { Sanctum::actingAs(User::factory()->create()); $article = Article::factory()->make();
$this->assertDatabaseCount('articles', 0);
$this->graphQL(' mutation ($slug: String!, $title: String!, $body: String!) { createArticle(input: { slug: $slug, title: $title, body: $body }) { title slug + thumbnail } }', [ 'slug' => $article->slug, 'title' => $article->title, 'body' => $article->body, ] )->assertJson([ 'data' => [ 'createArticle' => [ 'title' => $article->title, 'slug' => $article->slug, + 'thumbnail' => Article::DEFAULT_THUMBNAIL_PATH ], ], ]);
$this->assertDatabaseHas('articles', [ 'title' => $article->title, ]); });
copy

We already ensured that creating an article without a thumbnail will work and a default value will be returned for it. Now we have to ensure that, when creating a new article with a thumbnail, it also works!

tests/Feature/ArticleTest.php

it('can create an article', function () {
Storage::fake('s3');
Sanctum::actingAs(User::factory()->create());
$article = Article::factory()->make();
$this->assertDatabaseCount('articles', 0);
$operations = [
'query' => /** @lang GraphQL */ '
mutation ($slug: String!, $title: String!, $body: String!, $thumbnail: Upload) {
createArticle(input: { slug: $slug, title: $title, body: $body, thumbnail: $thumbnail }) {
title
slug
thumbnail
}
}
',
'variables' => [
'slug' => $article->slug,
'title' => $article->title,
'body' => $article->body,
'thumbnail' => null,
],
];
$map = [
'0' => ['variables.thumbnail'],
];
$file = UploadedFile::fake()->image('avatar.jpg');
$response = $this->multipartGraphQL($operations, $map, [
'0' => $file,
])->assertJson([
'data' => [
'createArticle' => [
'title' => $article->title,
'slug' => $article->slug,
],
],
]);
Storage::assertExists(Str::remove('/storage/', $response['data']['createArticle']['thumbnail']));
$this->assertDatabaseHas('articles', [
'title' => $article->title,
]);
});
copy

To run our suite now just use sail and your tests should pass! 😎

sail test
copy

Pest Test Suite

Conclusionslink

It's been a while since I wanted to test MinIO and I think this was the perfect occasion for it. This was probably the third and last part of laragraph as I don't think I have more ideas to continue this series, but if you have any suggestion for part 4 feel free to reach out to me! πŸ™

As always, you can find the implementation for the entire article in this Pull Request!

If you found this article interesting, feel free to share it with your colleagues and friends, because you know... Sharing is caring!

Also, if you enjoy working at a large scale in projects with global impact and if you enjoy a challenge, please reach out to us at xgeeks! We're always looking for talented people to join our team πŸ™Œ

Tiago Sousa

Written by Tiago Sousa

Hey there, my name is Tiago Sousa and I'm a Fullstack Engineer currently working at xgeeks. I'm a technology enthusiast and I try to explore new things to keep myself always updated. My motto is definitely "Sharing is caring" and that's exactly why you are currently reading this!

Interested in collaborating with me?

Let's Talk
footer

All the rights reserved Β© Tiago Sousa 2022

memoji12

Designed with love by Mauricio Oliveira