GridFS

Loading...

Do curso por Johns Hopkins University

Ruby on Rails Web Services and Integration with MongoDB

287 classificações

Johns Hopkins University

Ruby on Rails Web Services and Integration with MongoDB

287 classificações

Curso 3 de 6 em Specialization Desenvolvimento Web com Ruby on Rails

In this course, we will explore MongoDB, a very popular NoSQL database and Web Services concepts and integrate them both with Ruby on Rails. MongoDB is a used to handle documents with a pre-defined schema which will give the developers an ability to store, process and use data using it’s rich API. The modules will go in-depth from installation to CRUD operations, aggregation, indexing, GridFS and various other topics where we continuously integrate MongoDB with RailsRuby. We will be covering the interface to MongoDB using the Mongo Ruby API and the Mongoid ORM framework (the MongoDB access counterpart to RDBMS/ActiveRecord within Rails). The last portion of the course will focus on Web Services with emphasis on REST, its architectural style and integration of Web Services with Rails. Core concepts of Web Services like request/response, filters, data representation (XML/JSON), web linking and best practices will covered in depth. This course is ideal for students and professionals who have some programming experience and a working knowledge of databases.

Na lição

Aggregation Framework, Performance, and Advanced MongoDB

In this module, we’re going to explore the fundamentals of the Aggregation framework in MongoDB. We will work on examples where you will process data records and return computed results. You will learn about and get to work on hands-on examples where you will be grouping values from multiple documents and performing a variety of operations on the grouped data to get a single result. We will look at a number of aggregation commands and paging. We will explore some advanced concepts like full text search, GridFS, Geospatial API, and wrap up the module with detailed demonstrations of all the capabilities presented in this module.

GridFS Demo 15:56

GridFS Demo 27:32

GridFS Demo 317:15

Geospatial Demo9:25

Conheça os instrutores

Kiran Chittargi
Adjunct Professor, Graduate Computer Science
Whiting School of Engineering

[MUSIC]

Hello, and welcome back.

In this lecture, we will look at GridFS, a quick introduction, high-level

architecture, some examples, we'll look at the GridFS collection, and metadata.

So what is GridFS?

Well, GridFS is a capability provided by MongoDB for storing and

retrieving large files and also primarily binary files.

So the advantage of GridFS is mostly when you have opaque,

non query-able data, like a binary file, audio/video images,

weather data, and they are good candidates for GridfFS.

Now, the data is stored within MongoDB collections, so the data, even though it's

binary and it breaks apart into chunks, it is still stored in MongoDB collection.

Now, the files are broken into multiple chunks, and

there is going to be a one to many relationship,

there will be one metadata file that'll store all the information about

the file itself, like the size, the number of chunks, the ID.

And then there's many relationship to multiple chunks.

Each of these chunks are a separate document with a maximum size

of 255 kilobytes, so we'll talk about the architecture a little more and

try to understand how they all fit together.

Let's take an example of four documents,10 MB,18 MB,

and then 10 and 18 with some binary data within them.

The first one is a straightforward 10 MB document,

which has normalized types as embedded.

Think of a movie with a bunch of actors in it, it's 10 MB,

all the data is within the document.

Great candidate to store as a straight as is document, just save it as it is.

Query the data and

you can display the contents back to the client through a driver application.

18 MB document of query-able data with normalized type as embedded.

Let's take the same movie example.

Right, now you have a lot more information with the role,

you have a role actor, actor ID,

actor biography, picture and now the data is becoming bigger and bigger, right.

But the problem is this is 18 MB's the data size.

We know that Mongo has a size limitation of 16 MB, right?

So this will be a problem because Mongol will complain if you try to

insert this data.

Now since the data is query-able, meaning it's still key value pair object arrays,

whatever, but it's still a key value pair.

A good design here might be,

you may want to take the actor information in my example or directors or

other nested data, right, and move it to a different collection and

then make them a one-to-one or a one-to-many, whatever the relationship.

It's going to be linked relationship.

So you will still have the role information and

the actor ID, through which you can go to the actor collection and get the data.

The third example is a 10 MB of image blob.

So it's a binary data.

But there is some query-able data stored as a regular document, right?

Because Mongo does support binary.

So you can just pretty much take the document, and

save it as a, you know, just a regular document.

Having said that, since you have blob of binary data in this 10 MB document,

you can save the query-able data in the metadata section, and then use GridFS

to break the image into multiple chunks, because that's another possibility.

But 18 MB document, the last example of an image, now this is a really,

a good candidate for GridFS for two reasons.

One, the size obviously exceeds 18 MB so you have the 16 MB limitation.

That will will force you to break the file apart, but

since there is no normalized type here, you can do a linking.

So, you really have to really go down the grid at the spot, and

put all credible data into the metadata section or the file section.

Yeah, so just to give you an example,

from a very high level of when GridFS will come in,

because Grid FS is completely independent of any other restriction that Mongo has.

16 MB's is one restriction that will force you to go down the GridFS path.

But if the document is query-able data, then you may still want to stick with

just non GridSF solution where we can maybe look at linking and

moving the normalized data into a different collection like the actor and

the movie and all that.

But if it's opaque binary data that cannot be queried,

then GridSF is a good candidate for you to go down that path.

So here is a basic architecture, if you will, of GridFS.

So as I said, if you take a file and then break that file into multiple chunks, and

there will be one metadata file which will store all the information about this file.

So it's a classic one to many relationship.

So the file will have all the number of chunks,

the size, the ID, and some other information we'll talk about.

And the chunks will be, again, a number of chunks and there

will be a relationship back here, that's the relationship you have saying hey,

I belong to this file, this chunk belongs to the same file.

And then this information is important because when you are assembling it all

together, you need to know which file this chunk belongs to.

So GridFS uses two collections so

the default collection names are fs.files and fs.chunks.

The fs.files is where the metadata of the files will go.

And that's the query-able data, it's still your key value pair that you can query on.

And the second one, the second collection is the fs.chunks collection.

This is where all the individual chunks will go into a document and

the maximum of size of which is 255 kilobytes.

The GridFS will reassemble the contents when you access the chunks.

So when you access the data back, it will put it all together.

It knows, simply because of the metadata file and

what chunk belongs to what file, the order in which they were broken apart.

All that will be magically factored into, and the reassembly will happen

completely independent and transfer into the client applications.

So this is the chunks collection field.

So if you have an ID, the unique Object ID of the chunk,

files_id, the ID of the parent document, n,

the sequence number of the chunk, and the actual payload which is a binary type.

So this is the core field of the chunks, right?

The unique ID, who you belong to, and the sequence of the data.

Now on the other hand, the file has some mandatory field and some optional field.

It also has an ID, the length, The chunksSize, uploadDate and md5.

So let's take a little more in depth look at each of these in this chart.

So _id's a unique field, right?

And then we have the length which is the size of the document in bytes.

chunkSize is the size of each chunk, the default size is 255KB.

uploadDate, the date the document was first stored by GridFS.

What hash has been used.

The actual name, the human-readable name of the document.

contentType, the MIME type basically.

The aliases, if any.

And the metadata, any additional information that you want to store.

We will be taking an in depth look into some of these when we do the demo.

The GridFS stand alone demo and alter the web demo.

Just to summarize, GridFS is used for storing large files,

primarily binary files, by breaking them into smaller chunks.

But again, it all depends on your application.

As I said, simply because a document is 20 megabytes doesn't mean you have to use

GridFS, understood that there is a 16MB limitation,

but there are other ways to solve the problem.

So it all goes back to, is the data query-able data?

Is it a binary data?

So based on that, you want to make a decision as to using GridFS or not.

So next we'll go into a demo.

We'll look at this GridFS.

Take a file and send it to the GridFS framework and

also we have a GridFS web demo where we'll upload an image.

Thank you.

Explore nosso catálogo

Registre-se gratuitamente e obtenha recomendações, atualizações e ofertas personalizadas.

O Coursera proporciona acesso universal à melhor educação do mundo fazendo parcerias com as melhores universidades e organizações para oferecer cursos on-line.

© 2018 Coursera Inc. Todos os direitos reservados.

Baixar na App Store

Baixar no Google Play