Draft page to collect my notes about a time-series forecasting system. A lot of overlap with Microprediction.

Scoring and rewards

If there are multiple participants, there needs to be a reward/punishment metric to transfer scores from bad predictors to good ones. This aspect might need some tweaking to get right.

Function to take a list of predictions and wager amounts, the correct result, and then produces new amounts where the wagers that were closer to the correct result earn a percentage from the wagers that were further away. Also the sum of the wagers should stay constant (zero-sum).

• Calculate the distance of each prediction from the correct result.
• Determine the total amount wagered.
• Calculate the proportion of each wager to the total wagered amount.
• Distribute the wagers from those further away to those closer to the correct result based on their distance and proportion.
• Ensure that the total amount of wagers remains constant.

def redistribute_wagers(predictions, wagers, correct_result):
    # Calculate the distance of each prediction from the correct result
    distances = [abs(prediction - correct_result) for prediction in predictions]
    
    # Calculate the total amount wagered
    total_wagered = sum(wagers)
    
    # Calculate the proportion of each wager to the total wagered amount
    proportions = [wager / total_wagered for wager in wagers]
    
    # Calculate the total distance to distribute the wagers proportionally
    total_distance = sum(distances)
    
    # Calculate the amount to be redistributed to each wager based on distance
    redistribution = [((total_distance - distance) / total_distance) * total_wagered for distance in distances]
    
    # Adjust the redistribution amounts to ensure zero-sum
    redistribution_adjusted = [redistribution[i] - (proportions[i] * sum(redistribution)) for i in range(len(wagers))]
    
    # Calculate the new wager amounts
    new_wagers = [wagers[i] + redistribution_adjusted[i] for i in range(len(wagers))]
    
    return new_wagers

# Example usage:
predictions = [3, 5, 7]
wagers = [100, 150, 200]
correct_result = 4
new_wagers = redistribute_wagers(predictions, wagers, correct_result)
print(new_wagers)

This feels similar to a softmax. I should investigate that.

Workflow

• How to collect/update/cancel and score predictions?

Data source ideas

• Dweet.io has a nice variety of sensor data.
  • https://dweet.io/get/stats -> List of recently active data sources.
  • Get them, push to ClickHouse.
• Weather, obviously. It’s very easily available, and has such a large impact on other useful things like energy use and the general mood of the population.
• METAR reports
  • https://aviationweather.gov/data/api/
• Ezoic Ad Revenue Index
• Exchange rates

Microprediction

• https://web.archive.org/web/20230306165735/http://api.microprediction.org/
• http://web.archive.org/web/20201129125905/http://config.microprediction.org/config.json

Delays

• 70 seconds
• 310 seconds
• 910 seconds
• 3555 seconds

225 points are sent as a prediction. Think of these as samples from the probability distribution that you predict.