{"id":7746,"date":"2024-03-04T11:30:27","date_gmt":"2024-03-04T11:30:27","guid":{"rendered":"https:\/\/cloudlogz.com\/training_and_placement\/?post_type=product&#038;p=7746"},"modified":"2024-07-04T04:32:58","modified_gmt":"2024-07-04T04:32:58","slug":"master-azure-databricks","status":"publish","type":"product","link":"https:\/\/cloudlogz.com\/training_and_placement\/product\/master-azure-databricks\/","title":{"rendered":"Azure Databricks-PySpark"},"content":{"rendered":"<section class=\"l-section wpb_row height_medium width_full\" id=\"overview\"><div class=\"l-section-h i-cf\"><div class=\"g-cols vc_row via_grid cols_1 laptops-cols_inherit tablets-cols_inherit mobiles-cols_1 valign_top type_default stacking_default\"><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><h2>Overview<\/h2>\n<p>We are going to work on 10 Realtime projects for sufficient hands-on experience which can be used in your resumes. This course fits both beginners and experienced professionals. I can guarantee no course better than this is available in Market at this reasonable price. You have an option to receive Internship Certificate from CloudLogz LLC, San Diego, CA, United States without any additional cost.<\/p>\n<\/div><\/div><\/div><\/div><\/div><\/div><\/section><section class=\"l-section wpb_row height_medium width_full\" id=\"learn\"><div class=\"l-section-h i-cf\"><div class=\"g-cols vc_row via_grid cols_1 laptops-cols_inherit tablets-cols_inherit mobiles-cols_1 valign_top type_default stacking_default\"><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><h2>You&#8217;ll Learn<\/h2>\n<\/div><\/div><div class=\"w-separator size_small\"><\/div><div class=\"g-cols wpb_row via_grid cols_2 laptops-cols_inherit tablets-cols_inherit mobiles-cols_1 valign_top type_default stacking_default\" style=\"--gap:3rem;\"><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><div class=\"w-iconbox iconpos_left style_default color_primary align_left no_text\"><div class=\"w-iconbox-icon\" style=\"font-size:2rem;\"><i class=\"material-icons\">insights<\/i><\/div><div class=\"w-iconbox-meta\"><h3 class=\"w-iconbox-title\">Azure Blob Storage<\/h3><\/div><\/div><\/div><\/div><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><div class=\"w-iconbox iconpos_left style_default color_primary align_left no_text\"><div class=\"w-iconbox-icon\" style=\"font-size:2rem;\"><i class=\"material-icons\">access_time<\/i><\/div><div class=\"w-iconbox-meta\"><h3 class=\"w-iconbox-title\">Azure Data Lake Storage Gen2<\/h3><\/div><\/div><\/div><\/div><\/div><div class=\"w-separator size_medium\"><\/div><div class=\"g-cols wpb_row via_grid cols_2 laptops-cols_inherit tablets-cols_inherit mobiles-cols_1 valign_top type_default stacking_default\" style=\"--gap:3rem;\"><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><div class=\"w-iconbox iconpos_left style_default color_primary align_left no_text\"><div class=\"w-iconbox-icon\" style=\"font-size:2rem;\"><i class=\"material-icons\">groups<\/i><\/div><div class=\"w-iconbox-meta\"><h3 class=\"w-iconbox-title\">Azure Active Directory<\/h3><\/div><\/div><\/div><\/div><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><div class=\"w-iconbox iconpos_left style_default color_primary align_left no_text\"><div class=\"w-iconbox-icon\" style=\"font-size:2rem;\"><i class=\"material-icons\">code<\/i><\/div><div class=\"w-iconbox-meta\"><h3 class=\"w-iconbox-title\">Azure Key Vault<\/h3><\/div><\/div><\/div><\/div><\/div><div class=\"w-separator size_medium\"><\/div><div class=\"g-cols wpb_row via_grid cols_2 laptops-cols_inherit tablets-cols_inherit mobiles-cols_1 valign_top type_default stacking_default\" style=\"--gap:3rem;\"><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><div class=\"w-iconbox iconpos_left style_default color_primary align_left no_text\"><div class=\"w-iconbox-icon\" style=\"font-size:2rem;\"><i class=\"material-icons\">design_services<\/i><\/div><div class=\"w-iconbox-meta\"><h3 class=\"w-iconbox-title\">Azure Databricks<\/h3><\/div><\/div><\/div><\/div><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><div class=\"w-iconbox iconpos_left style_default color_primary align_left no_text\"><div class=\"w-iconbox-icon\" style=\"font-size:2rem;\"><i class=\"material-icons\">how_to_reg<\/i><\/div><div class=\"w-iconbox-meta\"><h3 class=\"w-iconbox-title\">Azure IoT Hub<\/h3><\/div><\/div><\/div><\/div><\/div><div class=\"w-separator size_medium\"><\/div><div class=\"g-cols wpb_row via_grid cols_2 laptops-cols_inherit tablets-cols_inherit mobiles-cols_1 valign_top type_default stacking_default\" style=\"--gap:3rem;\"><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><div class=\"w-iconbox iconpos_left style_default color_primary align_left no_text\"><div class=\"w-iconbox-icon\" style=\"font-size:2rem;\"><i class=\"material-icons\">design_services<\/i><\/div><div class=\"w-iconbox-meta\"><h3 class=\"w-iconbox-title\">Azure Event Hub\/ Kafka<\/h3><\/div><\/div><\/div><\/div><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><div class=\"w-iconbox iconpos_left style_default color_primary align_left no_text\"><div class=\"w-iconbox-icon\" style=\"font-size:2rem;\"><i class=\"material-icons\">how_to_reg<\/i><\/div><div class=\"w-iconbox-meta\"><h3 class=\"w-iconbox-title\">PySpark, Structured Streaming, Auto Loader, Delta, Delta LIVE Tables<\/h3><\/div><\/div><\/div><\/div><\/div><div class=\"w-separator size_medium\"><\/div><\/div><\/div><\/div><\/div><\/section><section class=\"l-section wpb_row height_medium width_full\" id=\"structure\"><div class=\"l-section-h i-cf\"><div class=\"g-cols vc_row via_grid cols_1 laptops-cols_inherit tablets-cols_inherit mobiles-cols_1 valign_top type_default stacking_default\"><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><h2>Course Structure<\/h2>\n<\/div><\/div><div class=\"w-separator size_small\"><\/div><div class=\"w-tabs style_default switch_click accordion remove_indents\" style=\"--sections-title-size:1.5rem\"><div class=\"w-tabs-sections titles-align_none icon_chevron cpos_right\"><div class=\"w-tabs-section\" id=\"jd01\"><button class=\"w-tabs-section-header\" aria-controls=\"content-jd01\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">1.Apache Spark Architecture<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-jd01\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><ul class=\"pl-3\">\n<li class=\"mt-3\">Apache Spark Architecture<\/li>\n<li class=\"mt-3\">Spark&#8217;s internal details (Driver, Executor, task, stages, jobs etc.)<\/li>\n<li class=\"mt-3\">Spark Memory Allocation (Driver and Executor Memory Allocation)<\/li>\n<li class=\"mt-3\">Cluster Deployment Modes (Client, Cluster)<\/li>\n<li class=\"mt-3\">Narrow and Wide Transformations<\/li>\n<li class=\"mt-3\">Spark different configurations<\/li>\n<li class=\"mt-3\">OOM ERRORS and its common causes<\/li>\n<li class=\"mt-3\">Interview Questions based on Apache Spark Architecture<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"ufbe\"><button class=\"w-tabs-section-header\" aria-controls=\"content-ufbe\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">2.Introduction to Databricks<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-ufbe\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><ul class=\"pl-3\">\n<li class=\"mt-3\">Introduction to Databricks<\/li>\n<li class=\"mt-3\">Walkthrough of Databricks workspace<\/li>\n<li class=\"mt-3\">Different types of clusters and their uses.<\/li>\n<li class=\"mt-3\">Magic Commands, DBUTILS, Secret Scopes<\/li>\n<li class=\"mt-3\">Notebook Parametrization<\/li>\n<li class=\"mt-3\">Accessing Blob Storage\/ ADLS Gen 2 using notebook<\/li>\n<li class=\"mt-3\">Understanding of DBFS<\/li>\n<li class=\"mt-3\">Interview Questions based on session.<\/li>\n<li class=\"mt-3\">Assignments<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"z1f8\"><button class=\"w-tabs-section-header\" aria-controls=\"content-z1f8\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">3.ADLS Access using Databricks<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-z1f8\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><ul class=\"pl-3\">\n<li class=\"mt-3\">Accessing Blob Storage\/ ADLS Gen 2 using notebook<\/li>\n<li class=\"mt-3\">Mounting the Databricks workspace with ADLS Gen2<\/li>\n<li class=\"mt-3\">Interview Questions based on session.<\/li>\n<li class=\"mt-3\">Assignments<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"t486\"><button class=\"w-tabs-section-header\" aria-controls=\"content-t486\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">4.PySpark Data Processing<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-t486\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><ul class=\"pl-3\">\n<li class=\"mt-3\">\n<p class=\"mt-3\">PySpark Data Processing<\/p>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Different File Types and selecting the optimal file types.<\/li>\n<li class=\"mt-3\">Read different file formats (CSV, JSON, Parquet. etc.)<\/li>\n<li class=\"mt-3\">Different reading and writing options.<\/li>\n<li class=\"mt-3\">Interview Questions based on session.<\/li>\n<li class=\"mt-3\">Assignments to be completed within session.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"j6f0\"><button class=\"w-tabs-section-header\" aria-controls=\"content-j6f0\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">5.PySpark Transformations<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-j6f0\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><ul class=\"pl-3\">\n<li class=\"mt-3\">\n<p class=\"mt-3\">PySpark Transformations<\/p>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Wide<\/li>\n<li class=\"mt-3\">Narrow<\/li>\n<li class=\"mt-3\">Joins<\/li>\n<li class=\"mt-3\">Actions<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"v992\"><button class=\"w-tabs-section-header\" aria-controls=\"content-v992\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">6.Spark SQL<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-v992\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><ul class=\"pl-3\">\n<li class=\"mt-3\">\n<p class=\"mt-3\">Spark SQL<\/p>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Databricks Database<\/li>\n<li class=\"mt-3\">Global Temporary View, Temporary View, View<\/li>\n<li class=\"mt-3\">External Tables, Managed Tables<\/li>\n<li class=\"mt-3\">Joins<\/li>\n<li class=\"mt-3\">Interview Questions based on session.<\/li>\n<li class=\"mt-3\">Assignments<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"bc4c\"><button class=\"w-tabs-section-header\" aria-controls=\"content-bc4c\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">7.Lakehouse Architecture<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-bc4c\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><ul class=\"pl-3\">\n<li class=\"mt-3\">\n<p class=\"mt-3\">Lakehouse Architecture<\/p>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Delta Tables<\/li>\n<li class=\"mt-3\">Time Travel, History, Vacuum<\/li>\n<li class=\"mt-3\">DML operations on Delta Tables<\/li>\n<li class=\"mt-3\">CDC (Change Data Capture)<\/li>\n<li class=\"mt-3\">Incremental File Loading<\/li>\n<li class=\"mt-3\">Interview Questions based on session.<\/li>\n<li class=\"mt-3\">Assignments<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"ped3\"><button class=\"w-tabs-section-header\" aria-controls=\"content-ped3\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">8.Unity Catalog<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-ped3\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><ul class=\"pl-3\">\n<li class=\"mt-3\">\n<p class=\"mt-3\">Unity Catalog<\/p>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Benefits of Unity Catalog (Data Governance)<\/li>\n<li class=\"mt-3\">Data Lineage, Data Auditing<\/li>\n<li class=\"mt-3\">Upgrade Hive Meta store to Unity Catalog<\/li>\n<li class=\"mt-3\">Interview Questions on Unity Catalog<\/li>\n<li class=\"mt-3\">Assignments<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"v15e\"><button class=\"w-tabs-section-header\" aria-controls=\"content-v15e\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">9.Unity Catalog Continued<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-v15e\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><ul class=\"pl-3\">\n<li class=\"mt-3\">\n<p class=\"mt-3\">Unity Catalog<\/p>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Benefits of Unity Catalog (Data Governance)<\/li>\n<li class=\"mt-3\">Data Lineage, Data Auditing<\/li>\n<li class=\"mt-3\">Upgrade Hive Meta store to Unity Catalog<\/li>\n<li class=\"mt-3\">Interview Questions on Unity Catalog<\/li>\n<li class=\"mt-3\">Assignments<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"x3ce\"><button class=\"w-tabs-section-header\" aria-controls=\"content-x3ce\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">10.Structured Streaming<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-x3ce\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><ul class=\"pl-3\">\n<li class=\"mt-3\">\n<p class=\"mt-3\">Structured Streaming<\/p>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Different sources, IoT hub, Kafka etc.<\/li>\n<li class=\"mt-3\">Stateless\/Stateful Transformations<\/li>\n<li class=\"mt-3\">Output Modes (Complete, Update, Append)<\/li>\n<li class=\"mt-3\">Tumbling and Sliding Window<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"b64a\"><button class=\"w-tabs-section-header\" aria-controls=\"content-b64a\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">11.Structured Streaming Continued<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-b64a\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><ul class=\"pl-3\">\n<li class=\"mt-3\">\n<p class=\"mt-3\">Structured Streaming Continued.<\/p>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Watermarking<\/li>\n<li class=\"mt-3\">Stream Joins<\/li>\n<li class=\"mt-3\">Kafka Sink<\/li>\n<li class=\"mt-3\">Cosmos Sink<\/li>\n<li class=\"mt-3\">File Sink<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"k8d0\"><button class=\"w-tabs-section-header\" aria-controls=\"content-k8d0\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">12.Delta LIVE Tables<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-k8d0\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><ul class=\"pl-3\">\n<li class=\"mt-3\">\n<p class=\"mt-3\">Delta LIVE Tables<\/p>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Quality Checks<\/li>\n<li class=\"mt-3\">Data Quarantine<\/li>\n<li class=\"mt-3\">Structured Streaming using Delta LIVE Tables<\/li>\n<li class=\"mt-3\">Autoloader<\/li>\n<li class=\"mt-3\">Interview Questions<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"kba5\"><button class=\"w-tabs-section-header\" aria-controls=\"content-kba5\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">13.Pipeline Optimizations (100% Practical)<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-kba5\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><ul class=\"pl-3\">\n<li class=\"mt-3\">\n<p class=\"mt-3\">Pipeline Optimizations (100% Practical)<\/p>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Identify the root cause of performance issue (RCA)<\/li>\n<li class=\"mt-3\">Different options to tune the performance.\n<ul class=\"mt-3\">\n<li class=\"mt-3\">Skew\u2014Resolve Skew<\/li>\n<li class=\"mt-3\">Shuffle\u2014Reduce Shuffle<\/li>\n<li class=\"mt-3\">Spill\u2014Resolve Spill<\/li>\n<\/ul>\n<\/li>\n<li class=\"mt-3\">Caching\/ Delta Caching- Difference between both<\/li>\n<li class=\"mt-3\">Caching\/ Delta Caching- Difference between both<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"fd7f\"><button class=\"w-tabs-section-header\" aria-controls=\"content-fd7f\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">14.Azure Devops Databricks CI\/CD<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-fd7f\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><p><strong>Reserved for Project Work and Productionizing the Pipeline using CI\/CD.<\/strong><\/p>\n<\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/div><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><\/div><\/div><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><\/div><\/div><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><\/div><\/div><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><\/div><\/div><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><\/div><\/div><\/div><\/div><\/section><section class=\"l-section wpb_row height_medium width_full\" id=\"instructor\"><div class=\"l-section-h i-cf\"><div class=\"g-cols vc_row via_grid cols_1 laptops-cols_inherit tablets-cols_inherit mobiles-cols_1 valign_top type_default stacking_default\"><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><h2>Instructor<\/h2>\n<\/div><\/div><div class=\"w-iconbox iconpos_left style_default color_primary align_none no_title\"><div class=\"w-iconbox-icon\" style=\"font-size:2rem;\"><i class=\"far fa-linkedin\"><\/i><\/div><div class=\"w-iconbox-meta\"><div class=\"w-iconbox-text\"><p><span>LinkedIn &#8211; <a href=\"https:\/\/www.linkedin.com\/in\/ashok-kumar-ade\/\" target=\"_blank\" rel=\"noopener\">Ashok Kumar<\/a><\/span><\/p>\n<\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/section><section class=\"l-section wpb_row height_medium width_full\" id=\"request_demo\"><div class=\"l-section-h i-cf\"><div class=\"g-cols vc_row via_grid cols_1 laptops-cols_inherit tablets-cols_inherit mobiles-cols_1 valign_top type_default stacking_default\"><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><h2>Request Demo<\/h2>\n<\/div><\/div>\n<div class=\"wpcf7 no-js\" id=\"wpcf7-f7651-o1\" lang=\"en-US\" dir=\"ltr\">\n<div class=\"screen-reader-response\"><p role=\"status\" aria-live=\"polite\" aria-atomic=\"true\"><\/p> <ul><\/ul><\/div>\n<form action=\"\/training_and_placement\/wp-json\/wp\/v2\/product\/7746#wpcf7-f7651-o1\" method=\"post\" class=\"wpcf7-form init\" aria-label=\"Contact form\" novalidate=\"novalidate\" data-status=\"init\">\n<div style=\"display: none;\">\n<input type=\"hidden\" name=\"_wpcf7\" value=\"7651\" \/>\n<input type=\"hidden\" name=\"_wpcf7_version\" value=\"5.8.7\" \/>\n<input type=\"hidden\" name=\"_wpcf7_locale\" value=\"en_US\" \/>\n<input type=\"hidden\" name=\"_wpcf7_unit_tag\" value=\"wpcf7-f7651-o1\" \/>\n<input type=\"hidden\" name=\"_wpcf7_container_post\" value=\"0\" \/>\n<input type=\"hidden\" name=\"_wpcf7_posted_data_hash\" value=\"\" \/>\n<\/div>\n<p><span class=\"wpcf7-form-control-wrap\" data-name=\"your-name\"><input size=\"40\" class=\"wpcf7-form-control wpcf7-text wpcf7-validates-as-required\" autocomplete=\"name\" aria-required=\"true\" aria-invalid=\"false\" placeholder=\"Name\" value=\"\" type=\"text\" name=\"your-name\" \/><\/span><br \/>\n<span class=\"wpcf7-form-control-wrap\" data-name=\"your-email\"><input size=\"40\" class=\"wpcf7-form-control wpcf7-email wpcf7-validates-as-required wpcf7-text wpcf7-validates-as-email\" autocomplete=\"email\" aria-required=\"true\" aria-invalid=\"false\" placeholder=\"Email\" value=\"\" type=\"email\" name=\"your-email\" \/><\/span><br \/>\n<span class=\"wpcf7-form-control-wrap\" data-name=\"tel-326\"><input size=\"40\" class=\"wpcf7-form-control wpcf7-tel wpcf7-validates-as-required wpcf7-text wpcf7-validates-as-tel\" aria-required=\"true\" aria-invalid=\"false\" placeholder=\"Number\" value=\"\" type=\"tel\" name=\"tel-326\" \/><\/span><br \/>\n<span class=\"wpcf7-form-control-wrap\" data-name=\"text-951\"><input size=\"40\" class=\"wpcf7-form-control wpcf7-text\" aria-invalid=\"false\" placeholder=\"Location\" value=\"\" type=\"text\" name=\"text-951\" \/><\/span>\n<\/p>\n<p><input class=\"wpcf7-form-control wpcf7-submit has-spinner\" type=\"submit\" value=\"Submit\" \/>\n<\/p><p style=\"display: none !important;\" class=\"akismet-fields-container\" data-prefix=\"_wpcf7_ak_\"><label>&#916;<textarea name=\"_wpcf7_ak_hp_textarea\" cols=\"45\" rows=\"8\" maxlength=\"100\"><\/textarea><\/label><input type=\"hidden\" id=\"ak_js_1\" name=\"_wpcf7_ak_js\" value=\"96\"\/><script>document.getElementById( \"ak_js_1\" ).setAttribute( \"value\", ( new Date() ).getTime() );<\/script><\/p><div class=\"wpcf7-response-output\" aria-hidden=\"true\"><\/div>\n<\/form>\n<\/div>\n<\/div><\/div><\/div><\/div><\/section><section class=\"l-section wpb_row height_medium width_full\" id=\"live_Projects\"><div class=\"l-section-h i-cf\"><div class=\"g-cols vc_row via_grid cols_1 laptops-cols_inherit tablets-cols_inherit mobiles-cols_1 valign_top type_default stacking_default\"><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><h2>Live Project<\/h2>\n<\/div><\/div><div class=\"w-tabs style_default switch_click accordion has_scrolling\" style=\"--sections-title-size:1.5rem\"><div class=\"w-tabs-sections titles-align_none icon_chevron cpos_right\"><div class=\"w-tabs-section\" id=\"s6f0\"><button class=\"w-tabs-section-header\" aria-controls=\"content-s6f0\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">1.Movies Ratings Analysis<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-s6f0\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"w-image us_custom_3090c82c align_center\"><div class=\"w-image-h\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"552\" src=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a1-1024x552.png\" class=\"attachment-large size-large\" alt=\"\" srcset=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a1-1024x552.png 1024w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a1-600x323.png 600w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a1-300x162.png 300w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a1.png 1275w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/div><\/div><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><h4 class=\"vc_custom_heading\">Movies Ratings Analysis Architecture<\/h4>\n<h5 class=\"mt-3\">Analyze and implement the Lakehouse for below requirements: \u2013<\/h5>\n<ul>\n<li class=\"mt-2\">Show the aggregated number of ratings per year<\/li>\n<li class=\"mt-2\">Show the rating levels distribution.<\/li>\n<li class=\"mt-2\">Show the 18 movies that are tagged but not rated.<\/li>\n<li class=\"mt-2\">Focusing on the rated untagged movies with more than 30 user ratings, show the top 10 movies in terms of average rating and number of ratings.<\/li>\n<li class=\"mt-2\">What is the average number of tags per movie in tags? And the average number of tags per user? How does it compare with the average number of tags a user assigns to a movie?<\/li>\n<li class=\"mt-2\">Identify the users that tagged movies without rating them.<\/li>\n<li class=\"mt-2\">What is the predominant (frequency based) genre per rating level?<\/li>\n<li class=\"mt-2\">What is the predominant tag per genre and the most tagged genres?<\/li>\n<li class=\"mt-2\">What are the most predominant (popularity based) movies?<\/li>\n<li class=\"mt-2\">Top 10 movies in terms of average rating (provided more than 30 users reviewed them)<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"jecd\"><button class=\"w-tabs-section-header\" aria-controls=\"content-jecd\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">Market Campaign Analysis<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-jecd\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><p class=\"mt-3\">loudLogz is the marketing company which gets the data from various sources in different formats and would like to ingest the data into their data lake and perform the analysis below.<\/p>\n<ul class=\"pl-3\">\n<li style=\"list-style-type: none;\">\n<ul class=\"pl-3\">\n<li class=\"mt-2\">Analyze data for each campaign, date, hour, os_type &amp; value to get all the events with counts.<\/li>\n<li class=\"mt-2\">Analyze data for each campaign, date, hour, store_name &amp; value to get all the events with counts.<\/li>\n<li class=\"mt-2\">Analyze data for each campaign, date, hour, gender_type &amp; value to get all the events with counts.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h4 class=\"vc_custom_heading\">Market Compaign Analysis Architecture<\/h4>\n<\/div><\/div><div class=\"w-image us_custom_3090c82c align_center\"><div class=\"w-image-h\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"583\" src=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a2-1024x583.png\" class=\"attachment-large size-large\" alt=\"\" srcset=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a2-1024x583.png 1024w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a2-300x171.png 300w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a2-600x342.png 600w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a2.png 1275w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"l81f\"><button class=\"w-tabs-section-header\" aria-controls=\"content-l81f\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">Financials-Bank Loan Analysis<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-l81f\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><p class=\"mt-3\">We will be working with transactional data referring to loan transactions and customers from US Bank (a famous bank around the world). You have two requirements from different areas of the bank<\/p>\n<ul class=\"pl-3\">\n<li class=\"mt-2\">The Marketing team needs to have updated customer data to be able to contact them and make offers.<\/li>\n<li class=\"mt-2\">The Finance area requires us to have daily loan transactions complemented with customer drivers to be able to analyze them and improve the revenue.<\/li>\n<\/ul>\n<p class=\"mt-3\">To comply with the request, we are going to perform\u00a0<b>incremental loads\u00a0<\/b>and use Databricks delta tables.<\/p>\n<\/div><\/div><div class=\"w-image us_custom_3090c82c align_center\"><div class=\"w-image-h\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"542\" src=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a3-1024x542.png\" class=\"attachment-large size-large\" alt=\"\" srcset=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a3-1024x542.png 1024w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a3-300x159.png 300w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a3-600x318.png 600w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a3.png 1275w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/div><\/div><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><h4 class=\"vc_custom_heading\">Financials-Bank Loan Analysis<\/h4>\n<p class=\"mt-3\"><b>Key Take Aways:-\u00a0<\/b>Learn end to end pipeline development and leverage your understanding to implement delta lakehouse for incremental data loading.<\/p>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"g179\"><button class=\"w-tabs-section-header\" aria-controls=\"content-g179\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">On-Prem Cloud Data Migration<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-g179\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><p class=\"mt-3\">Adventure Works is Bi-Cycle company, and they are planning to migrate their data from On Prem SQL Server to Azure. They have a list of 31 Tables which needs to be migrated to Azure. Scenarios include Manufacturing, Sales, Purchasing, Product Management, Contact Management, and Human Resources.<\/p>\n<p class=\"mt-3\">Design a develop a pipeline as per below requirements: \u2013<\/p>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Data Pipeline should incrementally load the data from on prem to azure.<\/li>\n<li class=\"mt-3\">Design the pipeline in such a way that if we want to add or remove any table from the list of tables, no changes to the pipeline are required.<\/li>\n<li class=\"mt-3\">Optimize the performance of the Pipeline.<\/li>\n<li class=\"mt-3\">Make sure that data is consistent in on prem and azure.<\/li>\n<li class=\"mt-3\">Use Azure DevOps to deploy the pipeline to Test and Prod.<\/li>\n<\/ul>\n<\/div><\/div><div class=\"w-image us_custom_3090c82c align_center\"><div class=\"w-image-h\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"470\" src=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a4-1024x470.png\" class=\"attachment-large size-large\" alt=\"\" srcset=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a4-1024x470.png 1024w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a4-300x138.png 300w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a4-600x275.png 600w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a4.png 1275w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/div><\/div><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><h4 class=\"vc_custom_heading\">On-Prem to Cloud Data Migration Architecture Key Take Aways: &#8211;<\/h4>\n<p>Meta Data Driven Pipeline development using Data Factory and using Best Practices for data consistency and optimization.<\/p>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"qa80\"><button class=\"w-tabs-section-header\" aria-controls=\"content-qa80\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">HealthCare-Covid19 Data Analysis<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-qa80\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><p class=\"mt-3\"><span>The COVID-19 pandemic also known as coronavirus pandemic is the ongoing outbreak of coronavirus disease (COVID-19). It is caused by a coronavirus called severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2).The outbreak was identified in Wuhan, China, in December 2019. The World Health Organization declared the outbreak a Public Health Emergency of International Concern on 30 January, and a pandemic on 11 March. Common symptoms include fever, cough, fatigue, shortness of breath, and loss of smell. Complications may include pneumonia and acute respiratory distress syndrome. The time from exposure to onset of symptoms is typically around five days, but may range from two to fourteen days. There is no known vaccine or specific antiviral treatment. Primary treatment is symptomatic and supportive therapy.<\/span><\/p>\n<\/div><\/div><div class=\"w-image us_custom_3090c82c align_center\"><div class=\"w-image-h\"><img loading=\"lazy\" decoding=\"async\" width=\"975\" height=\"510\" src=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a5.png\" class=\"attachment-large size-large\" alt=\"\" srcset=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a5.png 975w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a5-300x157.png 300w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a5-600x314.png 600w\" sizes=\"auto, (max-width: 975px) 100vw, 975px\" \/><\/div><\/div><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><p class=\"mt-3\"><b>Key Learning: \u2013<\/b>\u00a0Implement end to end pipeline using Data Factory and process different format files coming from different sources.<\/p>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Azure Data Lake Storage Gen2<\/li>\n<li class=\"mt-3\">Azure Blob Storage<\/li>\n<li class=\"mt-3\">Key Vault<\/li>\n<li class=\"mt-3\">Data Factory<\/li>\n<li class=\"mt-3\">SQL Database<\/li>\n<li class=\"mt-3\">Data Flows Transformations<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"v444\"><button class=\"w-tabs-section-header\" aria-controls=\"content-v444\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">Supply Chain Management- Out of Stock and Out of Shelf Prediction<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-v444\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><p>Out of stock (OOS) is one of the biggest problems in Supply Chain Management for any store. This project shows how OOS can be solved with real-time data and analytics by using the<b>Databricks Lakehouse Platform<\/b>\u00a0to solve on-shelf availability in real time to increase retail sales. The project can also be used for supply chain solutions.<\/p>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Use real-time insights to rapidly respond to demand.<\/li>\n<li class=\"mt-3\">Drive more sales with on-shelf availability.<\/li>\n<li class=\"mt-3\">Scale-out your solution to accommodate any size operation.<\/li>\n<\/ul>\n<p class=\"mt-3\">A bit different from OOS issues are on-shelf availability (OSA) problems where inventory may be in the store, but it\u2019s not placed in a manner that makes it easily accessible to customers. A product may be in inventory, but the principal display may give the impression that the item is out of stock or in low quantity. Items may be on the shelf but not pulled forward in a manner that makes them easily viewable by customers. Product may be technically in inventory but in a backroom that\u2019s not accessible to customers. Regardless of the reason, OSA issues tend to lead to lost revenue for retailers.<\/p>\n<\/div><\/div><div class=\"w-image us_custom_3090c82c align_center\"><div class=\"w-image-h\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"426\" src=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a6-1024x426.png\" class=\"attachment-large size-large\" alt=\"\" srcset=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a6-1024x426.png 1024w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a6-300x125.png 300w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a6-600x250.png 600w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a6.png 1275w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/div><\/div><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><h5 class=\"mt-3\">Key Learnings<\/h5>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Use ADL Gen2 for Bronze, Silver and Gold Layer.<\/li>\n<li class=\"mt-3\">Data Factory- Ingest data from different sources(Data Sets, IR, Linked Service)<\/li>\n<li class=\"mt-3\">Databricks- Using py-Spark implement Lakehouse.<\/li>\n<li class=\"mt-3\">MlFlow\u2014Use of predefined ML Model.<\/li>\n<li class=\"mt-3\">Analyse OOS and OSA<\/li>\n<li class=\"mt-3\">ADF Pipeline Orchestration.<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"le36\"><button class=\"w-tabs-section-header\" aria-controls=\"content-le36\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">HealthCare-Detecting Adverse Drug Reaction Incidents<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-le36\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><p class=\"mt-3\">Adverse Drug Reaction Incidents (ADRI) are potentially very dangerous to patients and are top causes of morbidity and mortality. Many ADRIs are hard to discover as they happen to certain groups of people in certain conditions, and they may take a long time to expose. Healthcare providers conduct clinical trials to discover ADRI before selling the products but normally are limited in numbers. Thus, post-market drug safety monitoring is required to help discover ADRI after the drugs are sold on the market.<\/p>\n<p class=\"mt-3\">Less than 5% of ADRIs are reported via official channels and the vast majority is described in free-text channels: emails &amp; phone calls to patient support centers, social media posts, sales conversations between clinicians and pharma sales reps, online patient forums, and so on. This requires pharmaceuticals and drug safety groups to monitor and analyze unstructured medical text from a variety of jargons, formats, channels, and languages \u2013 with needs for timeliness and scale that require automation.<\/p>\n<\/div><\/div><div class=\"w-image us_custom_3090c82c align_center\"><div class=\"w-image-h\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"359\" src=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a7-1024x359.png\" class=\"attachment-large size-large\" alt=\"\" srcset=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a7-1024x359.png 1024w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a7-300x105.png 300w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a7-600x210.png 600w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a7.png 1275w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/div><\/div><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><p class=\"mt-3\">In this project, we show how to use Spark NLP\u2019s existing models to process conversational text and extract highly specialized ADRI and DRUG information, store the data in Lakehouse, and analyze the data for various downstream use cases, including<\/p>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Conversational Texts ADE Classification<\/li>\n<li class=\"mt-3\">Detecting ADE and Drug Entities from Texts<\/li>\n<li class=\"mt-3\">Analysis of Drug and ADE Entities<\/li>\n<li class=\"mt-3\">Finding Drugs and ADEs Have Been Talked Most<\/li>\n<li class=\"mt-3\">Detecting Most Common Drug-ADE Pairs<\/li>\n<li class=\"mt-3\">Checking Assertion Status of ADEs<\/li>\n<li class=\"mt-3\">Relations Between ADEs and Drugs<\/li>\n<\/ul>\n<h5 class=\"mt-3\">Key Take Aways: \u2013<\/h5>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Azure Data Lake Storage Gen2, Azure Data Factory, Azure Key Vault<\/li>\n<li class=\"mt-3\">Azure Databricks +Machine Learning- Use the prebuilt NLP Model for ADE Detection.<\/li>\n<li class=\"mt-3\">Medallion Architecture Implementation<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"i828\"><button class=\"w-tabs-section-header\" aria-controls=\"content-i828\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">Retail\/Ecommerce-Retail Margin Improvement Analytics Solution<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-i828\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><p class=\"mt-3\">RMI is the process of collecting and analyzing data from the processing of transactions at a retail store. When a customer checks out, the data from that transaction feeds into several categories: inventory, sales, product, customer, and staff.<\/p>\n<p class=\"mt-3\">With volatility in the market and narrowing margins in retail, RMI analytics is critical for retailers to ensure they are running their inventory management program as effectively as possible. If an RMI system stores and reports data about inventory, retailers can have a better idea of what they\u2019re selling, what they\u2019re storing and what isn\u2019t moving.<\/p>\n<\/div><\/div><div class=\"w-image us_custom_3090c82c align_center\"><div class=\"w-image-h\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"537\" src=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a8-1024x537.png\" class=\"attachment-large size-large\" alt=\"\" srcset=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a8-1024x537.png 1024w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a8-300x157.png 300w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a8-600x315.png 600w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a8.png 1275w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/div><\/div><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><p class=\"mt-3\">Get started with our Solution for Real-Time Retail-Margin-Improvement Analytics to improve in-store operations by: Real-time Retail-Margin-Improvement Analytics to improve in-store operations by:<\/p>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Rapidly ingesting all data sources and types at scale<\/li>\n<li class=\"mt-3\">Building highly scalable streaming data pipelines with Delta Live Tables to obtain a real- time view of your operation.<\/li>\n<li class=\"mt-3\">Leveraging real-time insights to tackle your most pressing in-store information needs.<\/li>\n<\/ul>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Key Take Aways: \u2013<\/li>\n<li class=\"mt-3\">Blob Storage Account<\/li>\n<li class=\"mt-3\">Azure Data Factory for Batch Processing<\/li>\n<li class=\"mt-3\">Azure Key Vault<\/li>\n<li class=\"mt-3\">Azure Data Lake Storage Gen2<\/li>\n<li class=\"mt-3\">Azure IoT Hub- Learn in action not just theory.<\/li>\n<li class=\"mt-3\">Azure Databricks for Transformations<\/li>\n<li class=\"mt-3\">Delta Live Implementation<\/li>\n<li class=\"mt-3\">Medallion Architecture Implementation<\/li>\n<\/ul>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"k1fa\"><button class=\"w-tabs-section-header\" aria-controls=\"content-k1fa\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">OTT Media Streaming Quality of Service<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-k1fa\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><p class=\"mt-3\">As traditional pay TV continues to stagnate, content owners have embraced direct-to-consumer (D2C) subscription and ad-supported streaming for monetizing their libraries of content. For companies whose entire business model revolved around producing great content which they then licensed to distributors, the shift to now owning the entire glass-to-glass experience has required new capabilities such as building media supply chains for content delivery to consumers, supporting apps for a myriad of devices and operating systems, and performing customer relationship functions like billing and customer service.<\/p>\n<p class=\"mt-3\">With most vMVPD (virtual multichannel video programming distributor) and SVOD (streaming video on demand) services renewing monthly, subscription service operators need to prove value to their subscribers every month\/week\/day (the barriers to a viewer for leaving AVOD (ad-supported video on demand) are even lower \u2013 simply opening a different app or channel). General quality of streaming video issues (encompassing buffering, latency, pixelation, jitter, packet loss, and the blank screen) have significant business impacts, whether it\u2019s increased subscriber churn or decreased video engagement.<\/p>\n<\/div><\/div><div class=\"w-image us_custom_3090c82c align_center\"><div class=\"w-image-h\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"345\" src=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a9-1024x345.png\" class=\"attachment-large size-large\" alt=\"\" srcset=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a9-1024x345.png 1024w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a9-300x101.png 300w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a9-600x202.png 600w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a9.png 1275w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/div><\/div><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><h4 class=\"vc_custom_heading\">OTT Media Streaming Quality of Service Architecture<\/h4>\n<p class=\"mt-3\">Key Take Aways: \u2013<\/p>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Blob Storage Account<\/li>\n<li class=\"mt-3\">Azure Data Factory<\/li>\n<li class=\"mt-3\">Azure Key Vault<\/li>\n<li class=\"mt-3\">Azure Data Lake Storage Gen2<\/li>\n<li class=\"mt-3\">Azure Databricks + Auto Loader<\/li>\n<li class=\"mt-3\">PySpark Structured Streaming<\/li>\n<li class=\"mt-3\">Machine Learning- Use the prebuilt ML Model for predictions.<\/li>\n<li class=\"mt-3\">Logic Apps for Email notification<\/li>\n<li class=\"mt-3\">Medallion Architecture Implementation<\/li>\n<\/ul>\n<p class=\"mt-3\">When you start streaming you realize there are so many places where breaks can happen and the viewer experience can suffer, whether it be an issue at the source in the servers on-prem or in the cloud; in transit at either the CDN level or ISP level or the viewer\u2019s home network; or at the playout level with player\/client<\/p>\n<\/div><\/div><\/div><\/div><\/div><div class=\"w-tabs-section\" id=\"jc60\"><button class=\"w-tabs-section-header\" aria-controls=\"content-jc60\" aria-expanded=\"false\"><h3 class=\"w-tabs-section-title\">Advertisement Viewability Predictions<\/h3><div class=\"w-tabs-section-control\"><\/div><\/button><div  class=\"w-tabs-section-content\" id=\"content-jc60\"><div class=\"w-tabs-section-content-h i-cf\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><p class=\"mt-3\">Spark SQL<\/p>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Programmatic Advertisement Bidding (PAB) is the process by which companies buy and place ads online through automated auctions. Programmatic Advertisement Bidding (PAB) takes the work out of advertising by making it possible for advertisers to place hundreds and thousands of ads online, often in less than a second, without needing to individually reach out to online publishers.<\/li>\n<li class=\"mt-3\">The value of PAB is that it creates greater transparency for both publishers and advertisers in the ad market:\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Publishers can better control their inventory and CPMs (cost per 1000 ad impressions)<\/li>\n<li class=\"mt-3\">Advertisers that leverage PAB can boost advertising effectiveness by only bidding on impressions that are likely to be\u00a0<b>viewed<\/b>\u00a0by a given user.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p class=\"mt-3\">We\u2019ll implement the following data pipeline for Advertisement Viewability Predictions:<\/p>\n<\/div><\/div><div class=\"w-image us_custom_3090c82c align_center\"><div class=\"w-image-h\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"357\" src=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a10-1024x357.png\" class=\"attachment-large size-large\" alt=\"\" srcset=\"https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a10-1024x357.png 1024w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a10-300x104.png 300w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a10-600x209.png 600w, https:\/\/cloudlogz.com\/training_and_placement\/wp-content\/uploads\/2024\/02\/a10.png 1275w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/div><\/div><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><h4 class=\"vc_custom_heading\">Advertisement Viewability Predictions Architecture<\/h4>\n<p class=\"mt-3\">Key Take Aways: \u2013<\/p>\n<ul class=\"pl-3\">\n<li class=\"mt-3\">Blob Storage Account<\/li>\n<li class=\"mt-3\">Azure Data Factory<\/li>\n<li class=\"mt-3\">Azure Key Vault<\/li>\n<li class=\"mt-3\">Azure Data Lake Storage Gen2<\/li>\n<li class=\"mt-3\">Azure Databricks + Auto Loader<\/li>\n<li class=\"mt-3\">pySpark Structured Streaming<\/li>\n<li class=\"mt-3\">Machine Learning- Use the prebuilt ML Model for viewability.<\/li>\n<li class=\"mt-3\">Medallion Architecture Implementation<\/li>\n<\/ul>\n<p class=\"mt-3\"><b>Viewability<\/b>\u00a0is a metric that measures whether an ad was seen by a user. This gives marketers a more precise measurement about whether their message appeared to users in a visible way. In this project, we demonstrate a process to predict viewability. Keep in mind, the more likely users are to see an ad, the higher the price a DSPs will want to place on a bid for that ad, because it is ultimately more valuable to the advertiser.<\/p>\n<\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/section>\n","protected":false},"excerpt":{"rendered":"<p>Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Vestibulum tortor quam, feugiat vitae, ultricies eget, tempor sit amet, ante. Donec eu libero sit amet quam egestas semper. Aenean ultricies mi vitae est. Mauris placerat eleifend leo.<\/p>\n","protected":false},"featured_media":7748,"comment_status":"open","ping_status":"closed","template":"","meta":[],"product_cat":[19],"product_tag":[28],"class_list":{"0":"post-7746","1":"product","2":"type-product","3":"status-publish","4":"has-post-thumbnail","6":"product_cat-databricks-pyspark","7":"product_tag-databricks-pyspark","9":"first","10":"instock","11":"sale","12":"sold-individually","13":"purchasable","14":"product-type-simple"},"_links":{"self":[{"href":"https:\/\/cloudlogz.com\/training_and_placement\/wp-json\/wp\/v2\/product\/7746","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cloudlogz.com\/training_and_placement\/wp-json\/wp\/v2\/product"}],"about":[{"href":"https:\/\/cloudlogz.com\/training_and_placement\/wp-json\/wp\/v2\/types\/product"}],"replies":[{"embeddable":true,"href":"https:\/\/cloudlogz.com\/training_and_placement\/wp-json\/wp\/v2\/comments?post=7746"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cloudlogz.com\/training_and_placement\/wp-json\/wp\/v2\/media\/7748"}],"wp:attachment":[{"href":"https:\/\/cloudlogz.com\/training_and_placement\/wp-json\/wp\/v2\/media?parent=7746"}],"wp:term":[{"taxonomy":"product_cat","embeddable":true,"href":"https:\/\/cloudlogz.com\/training_and_placement\/wp-json\/wp\/v2\/product_cat?post=7746"},{"taxonomy":"product_tag","embeddable":true,"href":"https:\/\/cloudlogz.com\/training_and_placement\/wp-json\/wp\/v2\/product_tag?post=7746"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}