vangav/vos_backend

on YouTubebuilt on top techtechrefcassandraapple's cassandra deployment has over 75,000 nodes storing over 10PB of dataplay frameworkplay framework powers linkedin's 500,000,000 members backendakkaakka helps power e-shopping for hundreds of millions of amazon membersdatastax driversdatastax processes over 1,000,000,000,000 cassandra requests for netflix per dayfully-loadedutilityoptionsbackend generatordesign in json and generate 90+% of the code for a service + several utilitiessecurityfacebook/google auth, oauth 2, transaction tokens, encryption, ...networkstwilio, mail gun, rest client, rest jobs, ...push notificationsios, android, ...geo servicesgeo grids, reverse geo coding, geo hashing, ...dispatcher/workerdispatchable queries, push notifications, emails, sms, ...and more ...periodic jobs, thread pools, facebook graph api, snow flake ids (twitter's ids), compression, client generator, exceptions, geometry, metrics, deep vangav mighty integration, ...great templatestemplateservicesfeaturesdb tablescode: totalcode: generatedinstagram7475267,715 + 190,996238,465 (92.17%)analytics writer & reader29467,715 + 10,25274,552 (95.62%)whatsapp37767,715 + 7,83474,408 (98.49%)geo server13367,715 + 2,69170,300 (99.85%)calculate sum11067,715 + 54468,258 (99.99%)roadmaptimeeffect5 minfirst service (vos_calculate_sum) generated and running20 minsecond service (vos_geo_server) generated and running5-7 daysfinished all templates: command of all utilities, multi-service backends1-2 weeksfull command of modifying/extending the underlying backendthereafterbest-in-class-level in implementing top class backend services, finishing a year's worth of work every monthunix operating system (e.g.: mac os, ubuntu, etc ...)java 8python 2.7.10 (only for using cassandra)to check python version type in the following command in a terminal session python2.7 -V and the output should be Python 2.7.10installing python on mac osdownload python for mac osopen python-2.7.10-macosx10.6.pkg and follow the installer stepsinstalling python on ubuntucd ~/Downloadswget https://www.python.org/ftp/python/2.7.10/Python-2.7.10.tgztar -zxvf Python-2.7.10.tgzsudo apt-get install build-essential checkinstallsudo apt-get install libreadline-gplv2-dev libncursesw5-dev libgdbm-dev libc6-dev libbz2-dev libsqlite3-dev tk-dev libssl-devcd ~/Downloads/Python-2.7.10/sudo ./configuresudo make altinstallpython2.7 --versionbasic javabasic cassandra (optional for using cassandra database) on YouTube5-10 min: this tutorial explains how to generate and use the first vangav backend servicevos_calculate_sum is a service that takes a two floats (a and b) request and returns a double (c) response representing the summation of a and binitcreate a workspace directory my_services - this is the directory to contain both of vos_backend and all the services generated using itdownload this vos_backend.zip project (from the green clone or download button up there) inside the workspace directory created in (1) and unzip itrename downloaded vos_backend-master to vos_backendgenerate a new servicecreate a new directory my_services/vos_calculate_sumcopy controllers.json from vos_backend/vangav_backend_templates/vos_calculate_sum/ to the directory my_services/vos_calculate_sum created in (1)open a terminal session and cd to my_services/vos_backend/tools_binexecute the command java -jar backend_generator.jar new vos_calculate_sum to generate the serviceenter y for using the config directory in order to use controllers.json for generatingenter n for generating a worker service (using workers is explained in a separate section)writing the service's logic codeoptionally for eclipse users: open eclipse and import vos_calculate_sum projectfile > import > general > existing projects into workspace > next > set "select root directory" to my_services > under projects make sure that vos_calculate_sum is selected > finishdouble check the java version used for compiling the project: right click the project > properties > java compiler > enable project specific settings > compiler compliance level > 1.7 or 1.8open class HandlerCalculateSum.java under package com.vangav.vos_calculate_sum.controllers.calculate_sum, method processRequest should be as follows in order to complete the request-to-response logic @Override protected void processRequest (final Request request) throws Exception { // use the following request Object to process the request and set // the response to be returned RequestCalculateSum requestCalculateSum = (RequestCalculateSum)request.getRequestJsonBody(); // set response's value ((ResponseCalculateSum)request.getResponseBody() ).set( requestCalculateSum.a + requestCalculateSum.b); }start the servicecd to my_services/vos_calculate_sumexecute the command ./_run.shtry it outopen an internet browser page and type http://localhost:9000/calculate_sum?a=1.2&b=2.3 - this returns 3.5play with a and b values in the request string in (1)try issuing an invalid request (e.g.: set a to "xyz", don't set b, ...) to get a sense of how the default error response looks like (error responses are explained in depth in a separate section)stop the servicein the terminal session where you started the service press control + d on YouTube1. expand calculate sum to calculatorshows how to add controllers (api entry points) before and after service generation | on YouTube2. project's contentsexplains the building blocks of vangav backend | on YouTube3. next example with database (geo server)generates a service that has a backend database and also uses the geo services utility | on YouTube4. generated service structureexplains the building blocks of a generated service | on YouTube5. service generator config structureexplains the building blocks of the config used to generate a service | on YouTube6. request objectwhen implementing a service's logic, the passed request object keeps all the request's information from start to finish; this tutorial shows how to use that object | on YouTube7. error responseexplains what happens whenever an error happens during request processing and how to manually return various types of error response | on YouTube8. using dispatcher - worker(s)shows why and how to us the dispatcher with one or more worker service(s) | on YouTube9. debuggingexplains how to start and use the debugger10. deploy on a production serveronce your service is ready for release, this tutorial has the step-by-step process till your service is up and running on a production server - as well as how to scale it up | on YouTube11. vangav backend error codesvangav backend detects various types of errors (e.g.: invalid request param, invalid generator config, wrong utility method arguments, unauthorized third-party authentication, invalid vangav mighty solution, ...); this tutorial lists all of vangav backend's error codes/sub-codes with reference to the code producing them for ease of tracing if you get one | on YouTube on YouTubejava clientshandles generating java clients and contains vangav backend's client frameworkbackend generatormanages all vangav backend generators (api, database client, worker and java client)cassandrahandles generating database clients, contains the framework for generated clients and handles all cassandra operationscompressioncontains different compression algorithmscontent: verifiction/formatting/generationhandles content (code, phriction-wiki, text, ...) checking, formatting and generation; vangav backend relies on this package to verify generation config and format generated code, scripts, wiki, ...data structures and algorithmshas data structures (e.g.: heap, kd-tree, tuples, ...) and algorithms (collections, matricies, strings, arrays, ...)dispatcher - workerhandles generating worker services, contains the framework for dispatchers/workers and handles all of their operationsexceptionsvangav exceptions are used to handle bad request and internal error; those exceptions can be returned to the client and loggable (in database, text files, ...)filessimplifies various types of files needed for vangav backend services like: properties, json config, images, http response files, ommiting comments, directory operations, ...geo serviceshandles reverse geo coding, geo grids and geo hashingidshandles uuid operations, sequential ids and twitter's snow flake idsmathhandles mathematical and geometric operations like: numeric, ranges, circles, line segments, straight lines, ...metricshandles distance, time, date and calendar operationsnetworkshas sync/async rest client, rest jobs, email clients, twilio messaging and download utilityplay frameworkhandles generating api code and contains the framework for generated vangav backend services; vangav backend services' backbonepropertieshandles loading and extracting data from properties filespublic apishandles fetching data from facebook graph api and car2go apipush notificationshandles building and sending apple and android notificationssecurityhandles authentication (facebook, google, oauth 2 and transaction tokens) and cryptography (asymmetric, hashing and two-way encryption)systemhandles fetching system info (cpu usage, free ram/disk, os type, number of cores, ...) and console operation (interactive, commands, ...)thread poolcontains latch threads, periodic jobs and the thread pools responsible for maintaining top performance for vangav backend services (in-memory threads, cassandra, dispatcher and rest client)vangav mightyhandles generating and binding vangav mighty solutions | on YouTubecalculate sumsimple service | on YouTube | but my phone was under 10%geo servera more advanced service with a databse and uses geo services utility | on YouTube | I tried to make it slow!whatsappservice oriented architecture (main + worker + analytics), multi-keyspace database and basic authentication | on YouTube | am I a hologram?vangav analyticsservice oriented architecture (writer + reader) and generic service design (handles any type of analytics) | on YouTube | it may be foolish to turn this down!instagramon YouTube | [in-spuh-rey-shuh n]covered topicsservice oriented architecture (main + dispense + jobs + worker + dash board)multi-keyspace databaseoauth 2 and facebook authenticationfacebook graph apirest jobsperiodic jobsvangav mightypush notificationslogginganalyticsclient generatortest and bots services