{"version":10,"randomSeed":"c4be4107aa74f978aa00ef1adad05321","graph":{"viewport":{"xmin":-2.5,"ymin":-2.216978843167702,"xmax":2.5,"ymax":2.216978843167702}},"expressions":{"list":[{"type":"text","id":"2","text":"an object embedding is effectively a learned model, with a parameter (a little vector of parameters) for each object to embed, tasked to predict whether two objects are linked"},{"type":"text","id":"4","text":"say the prediction is P(linked) = e^(-d^2), where d is the distance between the embedding-vectors"},{"type":"text","id":"10","text":"loss function is sum[-ln(P(actual_linked))] over all possible pairs (actual_linked is linked if they are linked, not-linked if they aren't)"},{"type":"text","id":"18","text":"which branchlessly expands to -sum[L * ln(P(linked)) + (1 - L) * ln(1 - P(linked))], where L is 0 or 1 based on whether they're linked"},{"type":"expression","id":"19","color":"#2d70b3","latex":"-\\sum_{ }^{ }\\left(L\\ln\\left(P\\left(L\\right)\\right)+\\left(1-L\\right)\\ln\\left(1-P\\left(L\\right)\\right)\\right)"},{"type":"expression","id":"20","color":"#388c46","latex":"-\\sum_{ }^{ }\\left(L\\ln\\left(e^{-d^{2}}\\right)+\\left(1-L\\right)\\ln\\left(1-e^{-d^{2}}\\right)\\right)"},{"type":"expression","id":"23","color":"#000000","latex":"-\\sum_{ }^{ }\\left(L\\ln\\left(e^{-\\left(x_{2}-x_{1}\\right)^{2}-\\left(y_{2}-y_{1}\\right)^{2}}\\right)+\\left(1-L\\right)\\ln\\left(1-e^{-\\left(x_{2}-x_{1}\\right)^{2}-\\left(y_{2}-y_{1}\\right)^{2}}\\right)\\right)"},{"type":"expression","id":"24","color":"#c74440","latex":"-\\sum_{ }^{ }\\left(L\\left(-\\left(x_{2}-x_{1}\\right)^{2}-\\left(y_{2}-y_{1}\\right)^{2}\\right)+\\left(1-L\\right)\\ln\\left(1-e^{-\\left(x_{2}-x_{1}\\right)^{2}-\\left(y_{2}-y_{1}\\right)^{2}}\\right)\\right)"},{"type":"expression","id":"32","color":"#6042a6","latex":"-\\sum_{ }^{ }\\left(\\left(1-L\\right)\\ln\\left(1-e^{-\\left(x_{2}-x_{1}\\right)^{2}-\\left(y_{2}-y_{1}\\right)^{2}}\\right)-L\\left(\\left(x_{2}-x_{1}\\right)^{2}+\\left(y_{2}-y_{1}\\right)^{2}\\right)\\right)"},{"type":"text","id":"22","text":"to optimise that, get the gradient"},{"type":"text","id":"33","text":"wrt x1:"},{"type":"expression","id":"34","color":"#000000","latex":"\\frac{2\\left(L-1\\right)\\left(x_{2}-x_{1}\\right)e^{-\\left(x_{2}-x_{1}\\right)^{2}-\\left(y_{2}-y_{1}\\right)^{2}}}{1-e^{-\\left(x_{2}-x_{1}\\right)^{2}-\\left(y_{2}-y_{1}\\right)^{2}}}+2L\\left(x_{2}-x_{1}\\right)"},{"type":"text","id":"36","text":"wrt x2:"},{"type":"expression","id":"37","color":"#2d70b3","latex":"-\\frac{2\\left(L-1\\right)\\left(x_{2}-x_{1}\\right)e^{-\\left(x_{2}-x_{1}\\right)^{2}-\\left(y_{2}-y_{1}\\right)^{2}}}{1-e^{-\\left(x_{2}-x_{1}\\right)^{2}-\\left(y_{2}-y_{1}\\right)^{2}}}-2L\\left(x_{2}-x_{1}\\right)"},{"type":"text","id":"40","text":"so gradient on x2 is negative gradient on x1"},{"type":"text","id":"42","text":"wrt y1:"},{"type":"expression","id":"43","color":"#c74440","latex":"\\frac{2\\left(L-1\\right)\\left(y_{2}-y_{1}\\right)e^{-\\left(x_{2}-x_{1}\\right)^{2}-\\left(y_{2}-y_{1}\\right)^{2}}}{1-e^{-\\left(x_{2}-x_{1}\\right)^{2}-\\left(y_{2}-y_{1}\\right)^{2}}}-2L\\left(y_{2}-y_{1}\\right)"},{"type":"text","id":"46","text":"to go further, i think we need actual data"},{"type":"folder","id":"50","title":"library of useful functions","collapsed":true},{"type":"text","id":"51","folderId":"50","text":"project, R^3 -> R^2"},{"type":"text","id":"53","folderId":"50","text":"project list, R^(3*n) -> R^3 -> R^2 -> (R^2)^n"},{"type":"text","id":"55","folderId":"50","text":"dot product, R^n -> R^n -> R"},{"type":"text","id":"57","folderId":"50","text":"cross product, R^3 -> R^3"},{"type":"text","id":"59","folderId":"50","text":"normalise vector, R^n -> R^n"},{"type":"text","id":"61","folderId":"50","text":"insert element as replacement, R^n -> N -> {R, R^2} -> R^n"},{"type":"text","id":"63","folderId":"50","text":"rotate around each axis, R^3 -> R^3"},{"type":"text","id":"67","folderId":"50","text":"extract bit i of n, R -> Z -> N"},{"type":"expression","id":"68","folderId":"50","color":"#2d70b3","latex":"B\\left(n,i\\right)=\\operatorname{floor}\\left(\\operatorname{mod}\\left(\\frac{n}{2^{i}},2\\right)\\right)"},{"type":"text","id":"69","folderId":"50","text":"merge bit-list, N^n -> N"},{"type":"expression","id":"70","folderId":"50","color":"#6042a6","latex":"B_{i}\\left(v\\right)=\\operatorname{total}\\left(2^{\\left[0...\\operatorname{length}\\left(v\\right)\\right]}v\\right)","hidden":true},{"type":"text","id":"71","folderId":"50","text":"sum bits at given positions to get a bitmask, N^n -> N"},{"type":"expression","id":"72","folderId":"50","color":"#2d70b3","latex":"M\\left(v\\right)=\\operatorname{total}\\left(2^{v-1}\\right)","hidden":true},{"type":"expression","id":"73","color":"#c74440","latex":"D=\\left[M\\left(\\left[2,3,4\\right]\\right),M\\left(\\left[1,5,6,7,8,9,10,11,12,13,14\\right]\\right),M\\left(\\left[9\\right]\\right),M\\left(\\left[7,9\\right]\\right),M\\left(\\left[5\\right]\\right),M\\left(\\left[\\right]\\right),M\\left(\\left[5\\right]\\right),M\\left(\\left[\\right]\\right),M\\left(\\left[9\\right]\\right),M\\left(\\left[10,13\\right]\\right),M\\left(\\left[5,13\\right]\\right),M\\left(\\left[10,13\\right]\\right),M\\left(\\left[9,10\\right]\\right),M\\left(\\left[7\\right]\\right)\\right]"},{"type":"expression","id":"76","color":"#6042a6","latex":"P\\to\\left[\\left(\\operatorname{random}\\left(\\right),\\operatorname{random}\\left(\\right)\\right)\\operatorname{for}k=K\\right]"},{"type":"expression","id":"77","color":"#000000","latex":"K=\\left[1...\\operatorname{length}\\left(D\\right)\\right]"},{"type":"expression","id":"81","color":"#2d70b3","latex":"R=\\left[0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,0,0,0,0,1,0,0,0,1,0,0,1,0,0,0,0,0,0,0,1,0,1,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1,1,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0\\right]"},{"type":"expression","id":"83","color":"#6042a6","latex":"R\\to\\left[B\\left(d,i\\right)\\operatorname{for}d=D,i=\\left[0...\\left(\\operatorname{length}\\left(D\\right)-1\\right)\\right]\\right]"},{"type":"folder","id":"123","title":"points to configure (hidden)","collapsed":true},{"type":"expression","id":"75","folderId":"123","color":"#388c46","latex":"P=\\left[\\left(-0.8142067513888056,-0.013356840088026222\\right),\\left(0.09578727978925851,-0.026997204288573703\\right),\\left(-1.692749017241768,0.8477362872863988\\right),\\left(-1.402140034157894,-0.4297855408731236\\right),\\left(0.3619541397343826,-0.7575937804209282\\right),\\left(1.8309058050820561,0.20888024148977863\\right),\\left(-0.512242831158465,-1.0107846388294117\\right),\\left(1.378231356358112,-1.2437089454030676\\right),\\left(-0.9253881997876159,0.8206494158535868\\right),\\left(0.22216650483180994,1.1084915028593731\\right),\\left(0.7558815200352433,-0.0505347476014978\\right),\\left(0.8480895619117199,1.299151409327893\\right),\\left(0.17619998561084932,0.9271242501701424\\right),\\left(-0.3229466373332476,-1.6797187005152012\\right)\\right]","showLabel":true,"label":"${K}","labelOrientation":"above_right"},{"type":"text","id":"79","text":"now we actually compute loss!"},{"type":"expression","id":"85","color":"#c74440","latex":"C\\left(i,j\\right)=R\\left[\\operatorname{length}\\left(D\\right)j+i\\right]"},{"type":"expression","id":"80","color":"#c74440","latex":"L\\left(p\\right)=-\\operatorname{total}\\left(\\left[\\left\\{i=j+1:0,\\left(1-C\\left(i,j\\right)\\right)\\ln\\left(1-e^{-\\left(p\\left[j+1\\right].x-p\\left[i\\right].x\\right)^{2}-\\left(p\\left[j+1\\right].y-p\\left[i\\right].y\\right)^{2}}\\right)-C\\left(i,j\\right)\\left(\\left(p\\left[j+1\\right].x-p\\left[i\\right].x\\right)^{2}+\\left(p\\left[j+1\\right].y-p\\left[i\\right].y\\right)^{2}\\right)\\right\\}\\operatorname{for}i=\\left[1...\\operatorname{length}\\left(D\\right)\\right],j=\\left[0...\\left(\\operatorname{length}\\left(D\\right)-1\\right)\\right]\\right]\\right)"},{"type":"expression","id":"119","color":"#c74440","latex":"L\\left(P\\right)"},{"type":"text","id":"91","text":"gradients for each pair"},{"type":"expression","id":"92","color":"#2d70b3","latex":"G_{xm}=\\left[\\left\\{i=j+1:0,\\frac{2\\left(C\\left(i,j\\right)-1\\right)\\left(P\\left[j+1\\right].x-P\\left[i\\right].x\\right)e^{-\\left(P\\left[j+1\\right].x-P\\left[i\\right].x\\right)^{2}-\\left(P\\left[j+1\\right].y-P\\left[i\\right].y\\right)^{2}}}{1-e^{-\\left(P\\left[j+1\\right].x-P\\left[i\\right].x\\right)^{2}-\\left(P\\left[j+1\\right].y-P\\left[i\\right].y\\right)^{2}}}+2C\\left(i,j\\right)\\left(P\\left[j+1\\right].x-P\\left[i\\right].x\\right)\\right\\}\\operatorname{for}i=\\left[1...\\operatorname{length}\\left(D\\right)\\right],j=\\left[0...\\left(\\operatorname{length}\\left(D\\right)-1\\right)\\right]\\right]"},{"type":"expression","id":"94","color":"#6042a6","latex":"G_{ym}=\\left[\\left\\{i=j+1:0,\\frac{2\\left(C\\left(i,j\\right)-1\\right)\\left(P\\left[j+1\\right].y-P\\left[i\\right].y\\right)e^{-\\left(P\\left[j+1\\right].x-P\\left[i\\right].x\\right)^{2}-\\left(P\\left[j+1\\right].y-P\\left[i\\right].y\\right)^{2}}}{1-e^{-\\left(P\\left[j+1\\right].x-P\\left[i\\right].x\\right)^{2}-\\left(P\\left[j+1\\right].y-P\\left[i\\right].y\\right)^{2}}}+2C\\left(i,j\\right)\\left(P\\left[j+1\\right].y-P\\left[i\\right].y\\right)\\right\\}\\operatorname{for}i=\\left[1...\\operatorname{length}\\left(D\\right)\\right],j=\\left[0...\\left(\\operatorname{length}\\left(D\\right)-1\\right)\\right]\\right]"},{"type":"text","id":"96","text":"total gradient on each point"},{"type":"expression","id":"97","color":"#c74440","latex":"#G_{x}=\\left[\\operatorname{total}\\left(G_{xm}\\left[\\left(\\left(k-1\\right)\\operatorname{length}\\left(D\\right)+1\\right)...k\\operatorname{length}\\left(D\\right)\\right]\\right)+\\operatorname{total}\\left(G_{xm}\\left[\\left(k+\\operatorname{length}\\left(D\\right)\\right)...\\left(k+\\left(\\operatorname{length}\\left(D\\right)-1\\right)\\operatorname{length}\\left(D\\right)\\right)\\right]\\right)\\operatorname{for}k=\\left[1...\\operatorname{length}\\left(D\\right)\\right]\\right]"},{"type":"expression","id":"99","color":"#388c46","latex":"#G_{y}=\\left[\\operatorname{total}\\left(G_{ym}\\left[\\left(\\left(k-1\\right)\\operatorname{length}\\left(D\\right)+1\\right)...k\\operatorname{length}\\left(D\\right)\\right]\\right)+\\operatorname{total}\\left(G_{ym}\\left[\\left(k+\\operatorname{length}\\left(D\\right)\\right)...\\left(k+\\left(\\operatorname{length}\\left(D\\right)-1\\right)\\operatorname{length}\\left(D\\right)\\right)\\right]\\right)\\operatorname{for}k=\\left[1...\\operatorname{length}\\left(D\\right)\\right]\\right]"},{"type":"text","id":"115","text":"screw it, this will work!"},{"type":"expression","id":"117","color":"#6042a6","latex":"h=0.0001","slider":{"hardMin":true,"hardMax":true,"min":"0","max":"1"}},{"type":"expression","id":"116","color":"#388c46","latex":"G_{x}=\\left[\\frac{L\\left(P+\\left[\\left\\{s=k:\\left(h,0\\right),\\left(0,0\\right)\\right\\}\\operatorname{for}s=\\left[1...\\operatorname{length}\\left(D\\right)\\right]\\right]\\right)-L\\left(P\\right)}{h}\\operatorname{for}k=\\left[1...\\operatorname{length}\\left(D\\right)\\right]\\right]"},{"type":"expression","id":"121","color":"#388c46","latex":"G_{y}=\\left[\\frac{L\\left(P+\\left[\\left\\{s=k:\\left(0,h\\right),\\left(0,0\\right)\\right\\}\\operatorname{for}s=\\left[1...\\operatorname{length}\\left(D\\right)\\right]\\right]\\right)-L\\left(P\\right)}{h}\\operatorname{for}k=\\left[1...\\operatorname{length}\\left(D\\right)\\right]\\right]"},{"type":"text","id":"100","text":"and the update"},{"type":"expression","id":"101","color":"#6042a6","latex":"S=\\left(P\\to P-0.05\\left(G_{x},G_{y}\\right)-\\left(\\operatorname{mean}\\left(P.x\\right),\\operatorname{mean}\\left(P.y\\right)\\right)\\right)"},{"type":"text","id":"110","text":"connection visual"},{"type":"expression","id":"111","color":"#2d70b3","latex":"\\left[\\left\\{C\\left(i,j\\right)=1:\\operatorname{polygon}\\left(P\\left[i\\right],P\\left[j+1\\right]\\right),\\operatorname{polygon}\\left(\\left(0,0\\right)\\right)\\right\\}\\operatorname{for}i=\\left[1...\\operatorname{length}\\left(D\\right)\\right],j=\\left[0...\\left(\\operatorname{length}\\left(D\\right)-1\\right)\\right]\\right]"}],"ticker":{"handlerLatex":"S","minStepLatex":"1000","playing":true,"open":true}}}