What are fast, flat files or a MySQL RAM database?

Question

I need an easy way for multiple PHP scripts to exchange data.

Should I create a MySQL database with RAM storage mechanism and exchange data through it (can several scripts connect to the same database at the same time?)

Or would flat files with one piece of data per line be better?

Answer 1

Flat files? Nooooooo ...

Use a good database engine (MySQL, SQLite, etc.). Then, for maximum performance, use memcached for the cache .

---

Thus, you have the ease and reliability of exchanging data between processes using proven server software that processes concurrency, etc. But you get the storage speed of your data.

Keep in mind a couple of things:

• MySQL has a query cache. If you resubmit the same requests, you can get more performance without adding a cache layer.
• MySQL is very fast anyway. Are you under pressure to demonstrate that it is not fast enough?

Answer 2

Please do not use flat files for the convenience of staff.

If you just want to have shared data as quickly as possible, and you can keep it all in RAM, then memcached is the perfect solution.

If you want to save data, use a DBMS, for example MySQL.

Answer 3

A database is generally better, however, if you share a small, mostly static amount of data, there may be performance advantages (and simplicity) for this when using flat files.

Nothing but trivial data exchange, and I would choose DB.

Answer 4

1- If a flat file can be useful: A flat file can be faster than a database, but in very specific applications. They are faster if data is read from start to finish without any search or writing. If the data does not fit into memory and must be fully read in order to complete the task, it may be faster than the database. In addition, if there are many letters than reading, then a flat file will also be close, most database settings by default will have to make read requests wait for write completion to support indexes and foreign keys. Writing requests is usually slower than just reading.

TD / LR vesion: Use flat files for a job-based system (Aka, simple log analysis), and not for web search queries.

2- Flat files: If you go with a flat file, you will need to synchronize your scripts when the file is modified using the user lock mechanism. This can lead to a slowdown, corruption to a deadlock, if you have a mistake.

Database based Ram based databases? Most databases have query results in the cache, search indexes, which makes them very difficult for bits with a flat file. Since they are cached in memory, making it fully run from memory, most of the time is inefficient and dangerous. It is better to configure the database configuration correctly.

If you want to optimize performance with ram, I would first look at running your php scripts, html pages and small images from the ram drive. If the caching mechanism is more likely, it will be rude and will systematically manage the hard disk for unchanging static data.

An improved result can be achieved with the help of a load balancer, clustering with connecting the back plane to the bar-based SAN array. But that is the whole topic.

5- can several scripts connect to the same database at the same time?

Yes, its called a connection pool. In php (client side), its function opens a connection to its mysql-pconnect ( http://php.net/manual/en/function.mysql-pconnect.php ). I think you can configure the maximum open connection in php.ini. A similar setup on the server side of mysql determines the maximum number of concurrent client connections in the /etc/mysql/my.cnf file.

You must do this in order to take advantage of the processor's parallel process and avoid the php script to wait for each request to complete. This greatly increases productivity under heavy load.

There is also one connection pool / thread pool in the Apache configuration for regular web clients. See Httpd.conf.

Sorry for the text wall, it was boring. Louis.

Answer 5

If you use them on multiple servers, a file system approach will not shorten it (unless you have a consistent common file system, which is unlikely and may not be scalable).

Therefore, you will need a server database in any case to ensure the exchange of data between web servers. If you're serious about performance or availability, your application will support multiple web servers.

Answer 6

I would say that the MySql database would be the best choice if you do not have a mechanism for working with locks on flat files (and somehow for access control). In this case, the DB level (regardless of the specific DBMS) acts as a layer of indirection, allowing you not to worry about it.

Since the OP does not specify a web server (and PHP can actually work from the command line), I’m not sure that caching technologies are what they do here. The OP might look for some kind of conversion of flight data that is not controlled by the website. Who knows.

Answer 7

If your system has a PHP cache (which caches the compiled PHP code in memory, such as APC), try putting your data in a PHP file, like PHP code. If you have to write data, there are some security issues.

Answer 8

I need an easy way for multiple running PHP scripts to exchange data.

APC , and memcached are both good options depending on the context. shared memory may also be an option.

Should I create a MySQL database with RAM for storing and exchanging data through what (can I connect several scripts to the same database at the same time?)

This is also a decent option, but probably not as fast as APC or memcached.

Or would flat files with one piece of data per line be better?

If this is read-only data, this is an opportunity - but it can be slower than any of the above options. Especially if the data is large. However, instead of writing your own parsing code, just create a PHP array and include () the file.

If this is a data warehouse that can be accessed simultaneously by several authors, be sure to use a flat file! Writing to a flat file from several processes can lead to file corruption. You can lock the file, but you may encounter problems of blocking conflicts and a long wait time for the lock.

Concurrent write processing is the reason why applications such as mysql and memcached exist.