learning-it by example

#219 Data strategy 5.4 min

Choose AI Use Cases: Does Your Winner Survive?

Which AI project first? A scoring model picks a winner. I tested whether the winner survives a different opinion.

1. In the Python scoring model of eight made-up AI use cases, what did 2000 random weightings, with the scores unchanged, show about the baseline winner, support email triage?
2. In the Python weighted scoring model of eight made-up AI use cases, which use case came second in the baseline ranking, and with what score?
3. The Python weighted scoring model flips risk and effort so that low is good. What does a risk of three count as?
4. What can a weighted scoring model for choosing AI projects not see?
5. When should the weights of a weighted scoring model be set?

Practice only: nothing is saved and no score counts towards anything. Each answer comes with the reason.

  1. Score a ninth use case by hand

    Copy the code folder first. Add a ninth line to usecases.py with a use case of your own and scores from 1 to 10 for value, feasibility, risk and effort. Work out its score by hand with the weights in scoring.py, and predict its place in the ranking. Run python scoring.py to check. Undo this change afterwards, so the next tasks match the video.

    A hint

    Risk and effort count against a use case, so the code uses 11 minus the score for them.

  2. Make fraud flagging low risk

    In usecases.py change the risk and effort of Fraud flagging to 2 each. Predict where it lands in the ranking and whether it beats Support email triage. Run python scoring.py to check. Which change moved it up most, a lower risk or a lower effort? Undo this change afterwards, so the next tasks match the video.

    A hint

    Compare the weights in scoring.py. Risk and effort have the same weight.

  3. Weight value heavily

    In scoring.py set the weights to value 0.7 and 0.1 for each of the other three. Predict whether Support email triage stays first, and which high value use case rises. Run python scoring.py to check. What does this say about the winner? Put the weights back afterwards, so the next tasks match the video.

    A hint

    The four weights should add up to one. Look at which use cases have a high value score but low feasibility.

  4. Change the random seed

    In sensitivity.py change random.Random(7) to another whole number and run python sensitivity.py. Predict whether the same two use cases still come first and second, and whether the percentages repeat exactly. Then ask how much weight you would give a single run.

    A hint

    The seed only fixes which random numbers are drawn. The weights drawn are still spread over the whole range.

These tasks are for you to try on your own machine. Nothing is checked or saved.

What this does not show

Invented use cases and scores. Random weights treat every weighting as equally likely; with small weight changes the baseline winner stays first. A technique, not advice.

Up next in Data strategy

More in Data strategy

All Data strategy videos

Related videos