arrow_back 返回自由职业
F

MQTT-Hadoop Sales Data Pipeline Development

Freelancer

分享:
placeGH home_work远程 assignment合同制 public聚合职位 · GH

event发布于 2026年10月06日 · verified我们于 2026年10月06日 确认该职位仍然有效

这是您的公司吗?

US$ 250 – US$ 750 (每个项目)

职位介绍

Title: Urgent: MQTT to Hadoop Sales Data Pipeline with Hive Data Warehouse (Small Scope) We need an experienced Big Data developer to build a compact, end-to-end sales data pipeline. The scope is small and well defined, and we need it delivered quickly. We plan to award within hours to the right candidate. The project is fully self-contained. It needs no account access, credentials, invitations or files from our side. You can start right away in your own local environment. PROJECT OVERVIEW Build a small pipeline that ingests real-time sales events through MQTT, stores them in Hadoop, and organizes them in a Hive data warehouse for basic sales reporting. SCOPE OF WORK Part 1 - Data Generation (MQTT Publisher) Write a Python script that publishes simulated sales events to a local Mosquitto broker. Each event should contain order_id, timestamp, region, product, quantity and price. Part 2 - Data Ingestion (MQTT Subscriber to HDFS) Write a subscriber that consumes these events and writes them to HDFS. A single-node Hadoop setup is enough. Part 3 - Data Warehouse (Hive) Design a simple star schema in Hive with these tables: - fact_sales - dim_product - dim_region Load the ingested data into these tables. Part 4 - Sales Reporting Provide three SQL queries: - Total sales by region - Top 5 products by revenue - Daily sales totals DELIVERABLES - Complete source code (zip or rar) - A short README covering setup, execution steps and a brief description of the architecture - Screenshots showing the pipeline running and the query results TIMELINE Delivery is expected within 1 to 2 days of award. Please state your exact delivery time in your proposal.

REQUIREMENTS

Hands-on experience with Hadoop (HDFS, Hive) - Working knowledge of MQTT and Python - Understanding of data warehousing concepts (star schema) - Clear, timely communication HOW TO APPLY Begin your proposal with the word "PIPELINE" and include: - A brief summary of your relevant experience - Your proposed approach in 2 to 3 lines - Your delivery timeline Generic or copy-paste proposals will not be considered.

继续免费阅读

创建免费账户以查看完整职位信息并申请。

  • badge作品集对企业可见
  • notifications新职位邮件提醒
  • favorite始终免费,无任何隐藏条件