Rana Nasri Ghazzi

Empowering Data‑Driven Decisions That Bridge the Gap Between Insight and Action.

Category: Uncategorized

  • IT Survey Part 2

    Tools: Jupiter Notebook / Python / Pandas


    Objectives

    1. Identify the most popular programming languages among IT professionals.

    2. Analyze average salaries and income statistics for IT professionals.

    3. Explore the age distribution among IT professionals.

    4. Provide statistical information on working hours for part-time and full-time IT professionals.

    5. Determine the most popular databases among IT professionals.

    6. Examine the relationship between income and various factors such as working hours, age, education, and other variables.

    Tools:

    Python, Jupyter notebook

    Libraries

    numpy , pandas , seaborn , matplotlib.pyplot

    Dataset:

    Our data set is a survey works among IT professional , collected and published on Github in the link below.

    It has 11551 records and 84 columns. (11552, 85)

    1. Identify the most popular programming languages among IT professionals.

    There are unique 28 programming languages , the top most popular ones are:

    • JavaScript
    • HTML/CSS
    • SQL
    • Bash/Shell/PowerShell
    • Python

    2. Analyze average salaries and income statistics for IT professionals.

    3. Explore the age distribution among IT professionals.

    Summary statistics.

    • Mean: 30.77
    • Std (standard deviation): 7.37
    • Min: 16.00
    • 25% (first quartile, Q1): 25.00
    • 50% (median, Q2): 29.00
    • Max: 72.00
    • 75% (third quartile, Q3): 35.00

    4. Provide statistical information on working hours for part-time and full-time IT professionals.

    Reading into dataset:

    The Max of 1,012.00 hours is mathematically impossible (there are only 168 hours in a week). This indicates “noisy” data or data entry errors in the dataset.

    Similar to the full-time data, the Max of 375.00 hours is impossible for a single week, suggesting errors in the source data.

    Full-time workers are a much larger group in this data and center strictly around a 40-hour week. Part-time workers have a much broader distribution relative to their average, typically working between 20 and 35 hours.

    After applying outlier treatment:

    5. Determine the most popular databases among IT professionals.

    There are 13 unique databases , the top most popular ones are:

    MySQL

    Microsoft SQL Server

    PostgreSQL

    SQLite

    MongoDB

    6. Examine the relationship between income and various factors such as working hours, age, education, and other variables.

    Top 3 Insights From the Pair Plot Below:

    1. Experience Variables Are Strongly Correlated
    • YearsCode, YearsCodePro, and Age move together in clear linear patterns.
    • This confirms that experience‑related fields are consistent and reinforce each other.
    • Useful for feature engineering: you may want to avoid using all three in the same model due to multicollinearity.
    1. Compensation Shows Only a Weak Positive Trend With Experience
    • CompTotal increases slightly with YearsCodePro, but the scatter is wide.
    • This suggests compensation is influenced by many external factors (role, region, company size), not just experience.
    • Insight: experience alone is not a strong predictor of pay.
    1. Tool Knowledge Grows Slowly With Experience
    • Languages and Databases show mild upward trends with experience.
    • Most respondents know only a few tools, with a small number of outliers.
    • This indicates that tool count is not a strong differentiator across the population.

  • IT Survey Part 1

    Tools: Jupiter Notebook / Python / Pandas

    Objectives

    1. Identify the most popular programming languages among IT professionals.

    2. Analyze average salaries and income statistics for IT professionals.

    3. Explore the age distribution among IT professionals.

    4. Provide statistical information on working hours for part-time and full-time IT professionals.

    5. Examine the relationship between income and various factors such as working hours, age, education, and other variables.

    6. Determine the most popular databases among IT professionals.

    Tools:

    Python, SQL, Postgres

    Libraries:

    numpy , pandas , seaborn , matplotlib.pyplot

    Dataset:

    Our data set is a survey works among IT professional , collected and published on Github in the link below.

    It has 11551 records and 84 columns. (11552, 85)

    Technical Objectives:

    1. Connecting to API and import file

    2. Save file into Data frame.

    3. Cleaning the data (Nulls, outliers, rename, datatypes..)

    4. Normalize the data frame and create data model where languages field moved into new data frame, so that include one piece of information per record each.

    5. Transfer the new dataset into relational database or cloud.

    6. Performing EDA

    7. Visualizing & Plotting

    Salaries: statistics for IT professionals.

    The steps below were followed to clean the data and obtain a subset suitable for answering the compensation question:

    • Removing duplicates, filter columns, remove nulls, and split the employment field.
    • Creating a new column “earning” to standardize pay on an annual basis.
    • Filtering the dataframe to including only USD compensations for full-time employees, excluding nulls and zeros, and remove outliers by completing the top and bottom 5% with median .

    count 3,125.00
    mean 152,928.26
    std 207,792.90
    min 38,272.00
    25% 84,000.00
    50% 106,000.00
    75% 140,000.00
    max 1,664,000.00

    1. Central Tendency (The “Middle”)

    • Mean (152,928.26): The arithmetic average of all values.
    • 50% / Median (106,000.00): The exact middle value when the data is sorted.
    • Interpretation: Because the mean is much higher than the median, your data is right-skewed (positively skewed). This typically occurs when a few very large values pull the average up, while most data points remain lower. 

    2. Spread and Variability

    • Std (Standard Deviation: 207,792.90): This measures how far, on average, data points are from the mean. A standard deviation larger than the mean itself indicates extremely high variability and inconsistency.
    • Min (38,272.00) & Max (1,664,000.00): These define the total range. The massive gap between the max and the 75th percentile (140k vs. 1.66M) confirms the presence of significant outliers at the high end. 

    3. Distribution Percentiles (Quartiles)

    • 25% (84,000.00): One-quarter of your data is below this value.
    • 75% (140,000.00): Three-quarters of your data is below this value, meaning the top 25% of your data starts at 140,000.
    • Interquartile Range (IQR): The middle 50% of data falls between 84,000 and 140,000. 

    Summary Insight

    Most of your data (75%) is relatively clustered between 38,272 and 140,000. However, the high-end outliers—reaching up to 1.66 million—are so extreme that they have nearly doubled the “average” (mean) relative to the “typical” (median) value. When reporting on this data, the median (106,000) is likely a more accurate representation of a “typical” entry than the mean. 

    UseCase1Download

  • Behind Data…

    What lies behind data are facts— the story hides between the lines. Beneath every line there is opportunity to surface—a need for an address, a problem demanding attention. What data tells are stories, but the narrative is in your hands. It can go as far as you dare to go, but remember to carry the facts with you and keep them grounded in where they belong—in the story. You feed imagination with facts, and imagination is founded on them. And who knows… it might even start a new history.

    Rana

Social Media Auto Publish Powered By : XYZScripts.com