Geek Answers Handbook

Neural network structure

I am currently creating a neural network library. I just built it as an object graph for simplicity. I am wondering if anyone can evaluate the effectiveness of using an array based approach. What works for me now is very well suited for building networks that are close to arbitrary complexity. Regular (backpropped) networks as well as repeating networks are supported. I am considering creating a “compilation” of networks in several “simpler” forms, such as arrays.

I just wanted to find out if anyone has any practical advice or experience in creating neural networks that are well implemented in production. Is there any use to having the final product be based on an array and not based on a graph of objects?

PS Memory is less important than speed.

+4
source share
5 answers

It all depends on what language you use. I assume that you are using derivative C.

In my implementations, I found the object graph approach much higher. There is some tradeoff in speed, but ease of maintenance outweighs the challenges of finding objects. It all depends on whether you are looking for learning speed or deciding speed ... I assume that you are most worried about learning speed?

If necessary, you can eventually automatically optimize some problems with calling objects.

Given your secondary motive for network subnets, I think it’s even more important to be objective - it makes it easier to extract some of the work.

+2
source

People started using GPGPU techniques in AI, and having your neural network in matrix form could use the much faster matrix OSs in your typical graphics card.

+3
source

However, you implement it, you should never forget:

Just make sure you don’t have it maximize instead of minimize.

http://xkcd.com/534/

+2
source

Some time has passed, but I remember that speed is usually a problem only when training a neural network.

+1
source

I have no personal experience writing such a library, but I can connect you with some popular open source projects that you could learn from. (Personally, I would just use one of these existing libraries.)

0
source

More articles:


All Articles