Gaghiel Tutorial (Music Preferences)

In this tutorial we are going to create a recommendation system that is going to recommend new songs based on the previous songs that a user have heard. If you want to follow the tutorial step by step you can download the example data file in this link.

Angel Description

Our recommendation system Gaghiel is based on the Ramiel similarity search engine, and therefore we are able to include several nearest neighbor predictors in the recommendation ensemble. A new level of recommendations is possible!


File Specification:

Our Gaghiel engine is waiting for tsv files with 3 types of columns. These columns are:

ID ID columns represent the key of an object.
ITEMS A list of nominal values separated by → ‘ ; ‘.
WEIGHTS The weight for each item separeted by → ‘ ; ‘.

The file should consider these specifications:

  • The file should be in tsv format, which is a tab separated values = ‘ \t ‘.
  • The items column and weights column are going to have a concatenated list of values. Both lists for items and weights are going to have the same length.
  • The separator of this list is going to be a semicolon = ‘ ; ‘.
  • We do not expect to have a header.
  • The IDs can be nominal or numeric values.
  • Item values could be nominal or numeric values.
  • The weights must be numeric values (we are going to parse them as double).
  • We are going to read all the files in the folder, all of them should follow this format.
  • TSV Quote Character = ‘ ” ‘.
  • TSV Line End = ‘ \n ‘.
  • TSV Escape Character= ‘ \ ‘.

Example:

This file contains data about musical groups and artists that a given person listens to, and the number of times listened. The table shows fifteen lines of the file.

ID ITEMS WEIGHTS
00000c289a1829a808ac09c00daf10bc3c4e223b betty blowtorch;die Ärzte;melissa etheridge;elvenking;juliette & the licks;red hot chili peppers;magica;the black dahlia murder;the murmurs;lunachicks;walls of jericho;letzte instanz;goldfrapp;horrorpops;the butchies;jack off jill;babes in toyland;dropkick murphys;all:my:faults;le tigre;schandmaul;edguy;maximum the hormone;all ends;jack johnson;eluveitie;rasputina;london after midnight;the killers;mutyumu;judas priest;rob zombie;the bosshoss;blue Öyster cult;sandra nasic;john mayer;sleater-kinney;the who;disciple;tanzwut;guano apes;the rolling stones;little big town;team dresch;betty;l7;bif naked;girlschool;the wallflowers 2137;1099;897;717;706;691;545;507;424;403;393;387;361;358;329;316;310;302;288;281;244;232;231;229;227;222;220;210;208;205;198;198;189;185;183;182;175;168;154;153;151;150;145;137;135;135;134;134;131
00001411dc427966b17297bf4d69e7e193135d89 the most serene republic;stars;broken social scene;have heart;luminous orange;boris;arctic monkeys;bright eyes;coaltar of the deepers;polar bear club;the libertines;death from above 1979;owl city;coldplay;okkervil river;jim sturgess;deerhoof;fear before the march of flames;breathe carolina;mstrkrft;arcade fire;the strokes;cancer bats;the honorary title;michio kurihara;mgmt;queen;the secret handshake;animal collective;heavy heavy low low;hot hot heat;hadouken!;the wombats;ai aso;boris with michio kurihara;uffie;the last shadow puppets;boys noize;built to spill;nujabes;the rocket summer;little dragon;boris with merzbow;the album leaf;the microphones;simian mobile disco;m.i.a.;death cab for cutie;patrick wolf;the fiery furnaces;digitalism 12763;8192;6413;5361;5120;5110;4811;4283;4143;4091;3916;3315;3147;2658;2641;2427;2369;2362;2259;2102;1899;1854;1703;1682;1530;1468;1404;1384;1373;1323;1312;1254;1244;1232;1034;977;961;748;731;707;691;672;636;622;606;603;589;587;564;543;505
00004d2ac9316e22dc007ab2243d6fcb239e707d current 93;coil;andrew liles;six organs of admittance;16 horsepower;angels of light;the legendary pink dots;charalambides;festival;fern knight;nick cave & the bad seeds;fovea hex;amanda rogers;lux interna;nalle;antony and the johnsons;marissa nadler;a silver mt. zion;einstürzende neubauten;marc almond;mariee sioux;björk;bill fay;nurse with wound;baby dee;ghq;daniel higgs;flairck;jack rose;castanets;john jacob niles;the revolutionary army of the infant jesus;carter tutti;the one ensemble;orion rigel dommisse;jocelyn pook;matmos;ulver;ane brun;soisong;a hawk and a hacksaw;frank london;michael gira;larkin grimm;pelt;jean parlette 853;567;248;242;225;224;204;193;167;137;135;121;119;114;110;107;107;106;105;105;100;99;92;88;71;71;67;66;66;64;57;57;56;55;54;53;53;52;51;51;45;45;44;43;41;40
000063d3fe1cf2ba248b9e3c3f0334845a27a6bf opeth;cannibal corpse;elis;amoral;amon amarth;riverside;dark tranquillity;atrocity;kromlek;kamelot;novembers doom;poe;dido;stone temple pilots;audioslave;katatonia;ark;circus maximus;dimmu borgir;death;månegarm;cynic;korpiklaani;my dying bride;brujeria;agalloch;alice in chains;johann sebastian bach;kalmah;farmakon;children of bodom;dragonforce;dredg;green carnation;pain of salvation;rhapsody of fire;the mars volta;tool;arch enemy;at the gates;compay segundo;ensiferum;naglfar;nevermore;old mans child;shade empire;system of a down;tristania 106;48;31;29;26;24;17;16;16;14;14;14;13;12;11;11;10;10;10;9;9;8;8;8;7;6;6;6;6;5;4;4;4;4;4;4;4;4;3;3;3;3;3;3;3;3;3;3
00007a47085b9aab8af55f52ec8846ac479ac4fe devendra banhart;boards of canada;cocorosie;aphex twin;animal collective;atmosphere;joanna newsom;air;portishead;massive attack;broken social scene;arcade fire;plaid;fog;prefuse 73;m83;the flashbulb;pavement;goldfrapp;amon tobin;sage francis;four tet;brother ali;max richter;why?;autechre;radiohead;susumu yokota;electrelane;neutral milk hotel;bent;vetiver;k-the-i???;the microphones;illogic;annie;broadcast;beastie boys;aesop rock;mf doom;the books;bibio;rachel’s;architecture in helsinki;bola;isan;múm;elliott smith;caribou 456;407;386;213;203;189;184;178;162;149;144;143;133;132;121;118;118;117;109;106;106;100;96;94;91;86;86;85;84;83;81;80;78;77;76;75;75;74;72;71;70;69;69;68;68;68;67;66;65
0000c176103e538d5c9828e695fed4f7ae42dd01 radiohead;the hold steady;mogwai;boards of canada;justice;bright eyes;four tet;amon tobin;sigur rós;clutch;white denim;the go! team;the beatles;2 many dj’s;tv on the radio;tool;kyuss;dj shadow;air;a tribe called quest;the cinematic orchestra;beck;bon iver;röyksopp;rex the dog;bonobo;deadmau5;the decemberists;earth;snow patrol;battles;metronomy;dj format;the prodigy;lemon jelly;pink floyd;rjd2;the flaming lips;alasdair fraser & paul machlis;passion pit;holy fuck;michael jackson;mgmt;the rolling stones;late of the pier;flight of the conchords;simian mobile disco;muse;fleetwood mac;led zeppelin 2417;1955;1480;1115;1094;1039;1022;944;763;754;748;712;704;647;597;480;478;410;385;367;366;332;313;311;300;296;295;283;282;278;270;268;247;234;233;217;217;215;211;208;205;203;199;198;197;192;192;186;184;181
0000ee7dd906373efa37f4e1185bfe1e3f8695ae stam1na;dream theater;ac/dc;metallica;iron maiden;bob marley & the wailers;megadeth;children of bodom;trivium;raappana;nightwish;kotiteollisuus;circus maximus;sublime;viikate;volbeat;guns n’ roses;bon jovi;dio;soldiers of jah army;scorpions;teräsbetoni;dragonforce;nopsajalka;paramore;the darkness;soul captain band;asa;tarot;klamydia;gentleman;mötley crüe;queen;manowar;tool;killswitch engage;slayer;in flames;skid row;stone;bullet for my valentine;mokoma;pantera;brother firetribe;hurriganes;norther;stratovarius;kiuas;ozzy osbourne 758;572;554;473;471;447;255;247;244;227;189;160;156;136;136;133;125;101;101;101;97;95;89;77;75;73;68;67;63;62;61;59;57;54;54;53;52;51;46;46;44;44;44;42;40;40;40;39;34
0000ef373bbd0d89ce796abae961f2705e8c1faf pin;feel;elton john;orgy;brandy;pet shop boys;sade;belinda carlisle;metallica;eurythmics;mike oldfield;hall & oates;chris rea;laura branigan;the waterboys;maciej silski;leona lewis;george michael;black;frozen silence;sweet coffee;blackfield;cyndi lauper;pseudo echo;desireless;limp bizkit;modern talking;rihanna;a-ha;linkin park;hans zimmer;nelly furtado;madonna;blink-182;dave gahan;kelly clarkson;lucie silvas;natalie imbruglia;michelle branch;delta goodrem;new order;seal;atomic kitten;backstreet boys;mary j. blige;negative;justin timberlake;savage garden;björk;nerina pallot;erasure;goldfrapp;culture club;mr. oizo;robbie williams 421;401;140;128;105;101;101;97;95;85;79;76;73;65;64;62;59;56;53;49;49;47;47;47;39;38;37;36;35;34;32;31;29;25;24;23;23;23;22;21;21;21;20;20;20;20;19;19;17;17;16;16;15;14;13

 

This file has three specific columns. The first column is the ID of the person. The second column is a semi-colon separated list of the musical groups and artists listened by that person. And the last column is also a semi-colon separated list of the times that every group or artist has been listened by the person.

Note: the list in the second and last column have to be of the same length, this is because every item in the second column(musical group or artist) needs to have an associated weight in the last column (the number of times that person has listened to that group or artist).

The purpose of this tutorial is to recommend similar musical groups or artist based on a person’s previous music preferences.


Angel Parameters Specification:

These are the parameters needed for the angel creation:

Storage Units Specify the angel unit size reserved for creation.
Parallelism Specify the number of replications for the angel that you want to create.
Ramiel K Specify the number of results for the nearest neighbor search.
Pivots The number of primary search points in the engine.
Probability Minimum accepted probability for the results, any result with lower probability will be discarded.
Accepted Error Accepted search error from the distance calculated by the engine and the real distance.
Top Recommendations Number of recomendations that Gaghiel is going to return.

  • Create Folder

    • Click on “Create Folder” to create a container for your csv, tsv or json files that our similarity engine will search.

    • Provide the folder name and click on “Create Folder”.

    • Once the folder is created you will return to a folder list view.

  • Upload File(s)

    • In the “Folder” that you created click on “Upload File” to see the next modal.

      p4g

    • After choosing your files click on “Upload File”.

      p5g

    • You can see the progress bar while the file is being uploaded.

      p6g

    • Once the files are uploaded you will return to a folder list view.

      p7g

  • Create your Angel

    • Go to “Create Angel” section to choose the angel that works for your project.

      p8g

    • For this example we are going to create a Gaghiel. Click “Create” on Gaghiel image.

      p9g

    • The next step is choose the folder containing the files that you want to use to train the angel. Choose the folder to see a preview of the files.

    • The next step is to fill the Gaghiel parameters (default parameters will work fine) and to choose the name for your angel.

      p11g

      Click on “Create” to start the creation of the angel.

    • Once the creation started you can see a table with your current angels and the progress of the creation. When the state is running the angel is ready to answer queries.

  • Query Your Angel

    • For the query you have two options, Execute Query and Batch Query, both options can be accessed from “Your Angels” screen.

    • Execute Query

      Provide the values for the object that you want to query (items and weights separated by ‘ ; ‘), then click on “Execute Query”.

    • Other option is to use an object from a file to fill the query parameters, for this choose a folder containing the files that you want to use to query the angel, then click on “Fill Query Fields” and “Execute Query” to obtain your result.

    • Batch Query

      First choose a folder containing the files that you want to use to query the angel, and click on “Execute Query”, this is going to create a batch process.

    • Once the batch query has started you can see a table with your current batch files and the progress of the execution. When the state is completed the batch is ready for download.

Erick AlpizarGaghiel Tutorial Music Preferences