ofxAddons
This is a confirmed fork of ofxCaffe.
openFrameworks addon for visualizing and interfacing with pre-trained models in Caffe: Convolutional Architectures for Fast Feature Embedding. Requires Caffe, openFrameworks 64-bit, glog, hdf5, OpenCV, CUDA, pkmMatrix, and pkmHeatmap. Pre-trained models not included but can be found linked in Caffe's "Model Zoo" and placed in the bin/data directory of the example project.
Forks
Forks pushed to more recently than Geekrick88/ofxCaffe itself, plus any admin-confirmed fork with commits of its own — worth checking if you're hitting issues with the original.
README
ofxCaffe
Interface for Caffe: Convolutional Architectures for Fast Feature Embedding from BVLC.
Current Models
VGG ILSVRC 2014 (16 Layers): 1000 Object Categories VGG ILSVRC 2014 (19 Layers): 1000 Object Categories BVLC GoogLeNet: 1000 Object Categories Region-CNN ILSVRC 2013: 200 Object Categories (Region proposals not yet implemented) BVLC Reference CaffeNet: 1000 Object Categories BVLC Reference CaffeNet (Fully Convolutional) 8×8: 1000 Object Categories BVLC Reference CaffeNet (Fully Convolutional) 34×17: 1000 Object Categories MIT Places-CNN Hybrid (Places + ImageNet): 971 Object Categories + 200 Scene Categories = 1171 Categories
Instructions
(Warning: these probably won't work and will require edits/your help)
Install Caffe and all dependencies (-lglog -lgflags -lprotobuf -lleveldb -lsnappy -llmdb -lboost_system -lhdf5_hl -lhdf5 -lm -lopencv_core -lopencv_highgui -lopencv_imgproc -lcblas) Install openFrameworks 64-bit for osx I used this 1 year old branch which is about 1000 commits behind the master... I think the oF community is getting ready to release a stable 64-bit osx version??? Nick Hardeman's OSX 64-bit openFrameworks branch clone this repo into of_directory/addons/ofxCaffe clone [ofxOpenCv2461]() into of_directory/addons/ofxOpenCv2461 (could possibly be replaced by other opencv libraries, likely not the one that ships with openframeworks though; I've included OSX compiled OpenCV 2461 libraries in the addon/libs/opencv folder) clone pkmMatrix into of_directory/../pkm/pkmMatrix (This is OSX only due to its depency on Accelerate.framework; provides vectorized operations; can be replaced with OpenCV; please submit pull request and I wiil merge...) clone pkmHeatmap into of_directory/../pkm/pkmHeatmap (Converts grayscale images to RGB JET colormap using GPU) Go to the Caffe Model Zoo and download all necessary .caffemodel files into the bin/data directory (could make a script for this...; I know this last step is ridiculous. You have to go into the code and see what the names are and I probably changed the names too... well some of them anyways. I will make a script if there is enough interest...)
Example Project: Visualization
'1': Toggle predicted label output '2': Toggle layer parameters '3': Toggle layer outputs '4': Toggle probabilities graph '[' / ']': Change the current layer visualized '-' / '+': Change the current model '0': Toggle webcamera image
Troubleshooting
First make sure you can run Caffe and all tests (make runall) Check the Project.xcconfig defines and make sure they match up with where things should be (library files/source code)
To Do
Properly crop images and mirror them to produce batch images R-CNN region proposals Possibly other models can support region proposals and still be fast Add Network in Network Add Flickr style Fine Tuning Alternatives to visualizing layers? Deconvnets? Projections?