用 Python、Neo4j 和 ServiceNow 构建 GraphRAG 系统

Somewhere in your company's ServiceNow instance is the answer to the question an engineer asks at two in the morning: if this is broken, what else is about to break? Every fact needed to answer it has already been written down, correctly, by somebody doing their job properly. Getting it out still takes twenty minutes of opening one record at a time, and at the end you can't be sure the list is complete. This book is about closing that gap, and about measuring whether it really closes. You'll take a free ServiceNow developer instance, load a company's worth of servers, services, incidents, changes, problems, and knowledge into it, and read it back out with Python. Next, you'll model that estate as a graph, load it into Neo4j, and build eight different ways of choosing which records to put in front of a language model. Then you'll score all eight against thirty nine questions. I wrote and hashed those questions before any of the retrieval code existed, so nothing in the book could be tuned to them. Here's what you'll have at the end: Your own ServiceNow instance holding 11,891 configuration items and 68,900 tickets. The same estate as a Neo4j graph, with 28,694 dependency edges. Eight retrieval methods you built yourself, from plain keyword search to a walk through the graph. A language model answering from that retrieval, on a GPU you control, so the ticket text never leaves it. A results table saying which method actually found the right records, and a list of the fourteen things that table can't tell you. And here's what you'll learn along the way: What a graph database is for, and when it beats a relational one. How ServiceNow's CMDB stores dependencies, and why that makes a three hop question expensive. What retrieval means, and why it decides how good every answer is. How to build a comparison that could have proved you wrong. This isn't a victory lap. The question the book opens with is one that none of the eight methods answered, and Part 10 reports that with numbers instead of hiding it. You'll finish with a working system, and a real account of where it falls down. That's worth more than a demo that only ever gets asked the question it was built for. This book is free, start to finish. Every account it uses has a free tier, and the single rented GPU in Part 8 is priced in section 7 before you spend anything. Table of Contents Before You Start Part 0: The Problem, and Why a Graph Solves it 1. A Question Nobody Can Answer Quickly What's Real Here, and What's Written 2. Why This is Hard in ServiceNow Today 3. Why Plain Search Doesn't Solve it 4. The Four Questions This Book Answers 5. When You Shouldn't Build This 6. What You'll Build 7. What it Costs, in Dollars 8. How Long Each Part Takes 9. Who This is For 10. Three Ways Through This Book Part 1: Accounts and Keys, Created on Screen 11. Creating a ServiceNow Developer Instance 12. Waking a Sleeping Instance 13. Your Instance Login, and the Roles You Need 14. Creating an OAuth Application in ServiceNow 15. Creating a Neo4j Aura Account 16. Creating Aura API Credentials 17. The Aura Agent and MCP Credential, and What it's For 18. Creating an AWS Account and a User with the Right Permissions 19. Asking AWS for Permission to Use a GPU Server, Today 20. Setting a Spending Alarm Before You Launch Anything 21. Putting Every Key in One File Part 2: Getting Your Machine Ready 22. Which Python, and How to Check Yours 23. Getting the Code 24. Creating a Virtual Environment, and Why 25. Installing What You Need 26. A Note for Windows Readers 27. One Script That Connects to Everything and Prints Ok Part 3: The Dataset 28. What's In the Dataset 29. What's Real Here, and What Isn't How the Words Were Written, and Why it Matters to Part 10 30. Downloading the Dataset 31. Looking at it Before You Load it Part 4: Loading it into ServiceNow 32. Why We Add Data to ServiceNow First 33. The Obvious Way, One Record at a Time 34. Doing Several at Once 35. The Endpoint That Looks Built for This, and Isn't 36. Why it's Slow 37. The Fast Way, Running the Work Inside ServiceNow 38. When You Must Not Skip Those Rules 39. Loading Configuration Items is Different 40. Making the Loader Safe to Restart 41. Running it, and Checking What Landed Part 5: Reading it Back into Python 42. Installing Snowloader, and What it Does 43. Your First Query, and the Shape that Comes Back 44. Every Field Has Two Values 45. One Timestamp, Two Different Values 46. Reading the Dependency Table 47. Reading Work Notes, Which Aren't a Column 48. Paging, and What Happens When You Forget 49. Your Account May See Less Data Than Mine, with No Warning 50. Turning the Answers into Tables Part 6: Modeling ServiceNow as a Graph 51. Start from the Questions, Not the Tables 52. What ServiceNow Actually Gives You 53. Node, Relationship, or Property 54. Drawing the Model on Paper First 55. The Direction Trap 56. The Relationship That Points Both Ways 57. Never Key an Edge to the Words 58. A Configuration Item is Several Classes at Once 59. How Incidents Link to Configuration Items 60. Bringing Changes into the Graph 61. People and Groups 62. When a Date Should Be a Node 63. Items That Everything Else Connects to 64. Dependency Loops 65. How Fresh is This Edge? 66. Three Modeling Mistakes, and Why Each One is Wrong Part 7: Loading the Graph 66b. Start Here if You Only Want the Graph 67. Two Ways to Run Neo4j 68. Creating an Aura Instance in the Console 69. Creating One from the API Instead 70. Which Size You Need, with the Arithmetic 71. Running Neo4j in Docker 72. Constraints and Indexes, Before Any Data 73. Loading with UNWIND, and Why One Row at a Time is Slow 74. Loading the Relationships 75. Checking the Load 76. Seeing it in Neo4j Browser 77. Keeping it Up to Date Part 8: Running Your Own Model on Your Own GPU 78. Why Run Your Own Model at All? 79. Choosing the Model 80. Choosing the Embedding Model 81. Choosing the Server, with Real Prices 82. Launching it 83. Drivers and CUDA, and the Five Things That Go Wrong 84. Serving the Model with vLLM 85. Serving the Embedding Model 86. Calling Both From Your Laptop 87. Measuring it 88. Shutting it Down Properly Part 9: Five Ways to Retrieve 89. What Retrieval Means, Before Any Code 90. The Vector Index, and What it Physically is 91. How You Cut the Text into Chunks, and Why it Matters More Than Anything Else 92. Three Ways to Chunk this Data, Compared 93. Creating Embeddings and Storing Them 94. Creating the Vector Index 95. Retriever One: Pure Similarity 96. The Full Text Index, and Why Keyword Search is Still Good 97. Retriever Two: Similarity and Keywords Together 98. Retriever Three: Find by Similarity, Then Walk the Graph 99. Retriever Four: Both Indexes, Then Walk the Graph 100. Retriever Five: Let the Model Write the Query 101. Making a Written Query Correct, Not Just Safe 102. Keeping a Written Query Safe 103. Ticket Text Can Carry Instructions That Attack Your Model 104. Reordering Results Before Answering 105. Which Retriever Suits Which Question Part 10: Measuring Which One is Better 106. The Questions, Written Before the Graph Was Designed 107. Sorting Questions by Type 108. Did it Find the Right Records? 109. Making the Comparison Fair 110. Running All Eight 111. The Results 112. The Question Where Similarity Should've Won, and the Finding Underneath it 113. Changing the Chunking, and Running it All Again 114. Breaking the Dependency Data on Purpose 115. Speed and Cost 116. The Results Table, and What it's Allowed to Say 117. Running it Again with a Different Embedding Model 118. What to Build Next Thanks for Reading! Before You Start What You Need to Know: You'll need enough Python to read a script and run it: a for loop, a function call, a dictionary. You'll be reading and running the code in this book, not writing a framework. And you'll need enough knowledge of the command line to change directories, run a script, and read an error message. You don't need ServiceNow experience. Part 1 creates a free developer instance, and Part 3 explains every table before anything is loaded into it. You don't need Neo4j or Cypher either. Parts 6 and 7 teach both from nothing, and we'll define every term just below, before you meet it. Finally, you don't need a machine learning background. Parts 8 and 9 explain embeddings, tokens, and retrieval in plain English as they arrive. What You Do Need: This table covers what you will need to follow along: what where you set it up what it costs Python 3.10 or newer section 22 free A ServiceNow developer instance section 11 free Neo4j, either Aura's free tier or Docker section 67 free A GPU for one afternoon Part 8 about $5, priced in section 7 The GPU is the only thing here that costs money, and Part 8 is skippable. Section 19b lists the ways out of it. The measurements in Part 10 don't change if you use a hosted model instead, because retrieval happens before the model is involved. Part 0: The Problem, and Why a Graph Solves it 1. A Question Nobody Can Answer Quickly The time is 02:10. The payments service is failing. You're the engineer on call. Before you can fix anything, you need to know one thing: what else is about to break? The answer exists. It's sitting in ServiceNow right now. Somebody recorded that the payments service runs on an application. Somebody else recorded that the application uses a database. A third person recorded which storage array that database sits on. Every one of those facts was entered correctly, by a real person, doing their job properly. None of that helps you at 02:10. To get your answer, you open the payments service record. You read its dependencies. You open each one. You read its dependencies. You open each of those. Twenty minutes later you have a list on a notepad. You're not sure it's complete. The incident is still open. Here's what that walk is worth, on the estate this book ships with. An estate is everything a company owns and runs: its servers, services, and databases. This one holds 11,891 of them. Ask it upward first, meaning what stops working if payments stops. The answer is 16 items. Only 2 of those 16 appear on the payments service record itself. The other 14 are further away, each one reached by opening another record, and then another. Now ask it downward, meaning what underneath could be causing this. The payments service runs on an application called app0958. That application uses a database called pg0711. That database sits on a storage array called san-eu-west-01. That storage array carries 512 databases, belonging to 15 different teams: billing, catalogue, checkout, fraud, identity, inventory, loyalty, notifications, onboarding, payments, pricing, reporting, search, settlement, and shipping. So the real question at 02:10 isn't really about payments at all. Are you looking at one broken service? Or at the first symptom of something underneath that's about to stop 15 teams working? The records needed to answer that are all in ServiceNow. The array is three hops away. A hop is one step from a record to the record it points at. Three hops means four records to open, one after another. Each one tells you only where to look next. Nothing on the payments service record tells you the array exists. That's the problem this book takes on. Nobody caused it by doing anything wrong, and section 2 says what does cause it. The Three Tools, and What Each One Does Three tools sit between that problem and an answer, and they each do one job. ServiceNow is where the facts already are. Most companies use it to run IT. Every server, service, and database is a row in it. Every ticket is a row. Every dependency between two items is a row too. Nothing has to be collected. It's already written down. Neo4j is a graph database. It stores the same facts as circles joined by named arrows. In a graph the connections are the data, not something you rebuild every time you ask. Following an arrow costs the same whether you follow one or twenty. That's why three hops stops being a twenty minute job. GraphRAG is the last step. You ask in plain English. The graph picks which records matter. Those records go to a language model, and it writes the answer from them. The R in RAG is retrieval, which means choosing what to show the model. Choosing well is what most of this book is about. That's the whole book in one picture. On the left your facts sit in ServiceNow, one row each. Those aren't only configuration items: the book reads 60,000 incidents against 11,891 items, and it reads changes, problems and knowledge articles too. In the middle they become a graph in Neo4j. On the right you ask in plain English, and the answer is built from whatever the retrieval found. Parts 1 to 7 build the left and the middle. Parts 8 to 10 build the right, and Part 10 measures how often the retrieval returned the right thing. I want to be straightforward with you about the ending, here at the start. We'll build the whole thing. The records come out of ServiceNow and the graph goes up. We'll write and measure eight different ways of choosing what to show a model. Ask the graph this question directly and it answers in milliseconds. Part 7 shows exactly that. The step in between is what doesn't work yet. That's where a sentence in English has to become the right question for the graph. On this estate, with the questions frozen before the graph existed, not one of those eight ways answered the 02:10 question. Part 10 section 108b reports that zero alongside everything else. So read this as a build and a measurement, not a victory lap. You'll finish with a working system and an honest account of where it falls down. That's worth more than a demo that only ever gets asked the question it was built for. What's Real Here, and What's Written Every number in this book comes from one dataset, and it ships with the code. Part 3 walks through it file by file before you load any of it. Before you read another number, you should know which of them describe a real thing. Start with the real half. The ServiceNow instance is real: you create it yourself, and it's free. So are the tables, the fields, and the API. So is the field behaviour, including the parts the documentation doesn't mention. So are the identification engine, the business rules, and the rate limits. And so is every measurement in this book, taken on that instance and on this data. The written half is the estate itself. There's no company with these servers. The words inside the tickets are written too, every short description, every work note, and every resolution. They have to be written, and the reason is worth one paragraph. An incident's work notes contain hostnames, internal service names, customer names, and sometimes credentials pasted by an engineer in a hurry. It's some of the most sensitive text an organisation holds, and no company will ever publish it. That's why every public dataset in this space is either tiny or invented. It's also the reason this book runs its own model rather than calling a hosted API. If the text is the sensitive part, sending it to somebody else's service is exactly what a security review refuses. The dataset is generated by a seeded script that ships with the book. The Words You'll Need, Before You Meet Them Seventeen words carry the whole book. Here's each one in plain English, before anything below depends on it. Read it once now, and return to it whenever a word stops meaning something. A node is one thing, like a server, a service, or a ticket. It's drawn as one circle. A label is the graph's own name for what kind of thing a node is, like Server or Incident. One node can carry more than one. A relationship is a connection between two nodes, with a direction and a name. "This application runs on that server" is drawn as one arrow between two circles. A property is a fact stored on a node or a relationship, like a server's name or a ticket's priority. A graph is nodes and relationships together. That's the whole idea. What makes it useful is that following a relationship costs the same whether you follow one or twenty. Cypher is the language you'll use to ask a Neo4j graph a question. It's built around drawing the shape you want in text, and it looks more like a picture than like SQL. CMDB stands for Configuration Management Database. It's the part of ServiceNow that records what you own and how it's connected. CI stands for Configuration Item. It's one thing in the CMDB, like a server, a database, or a service. LLM stands for Large Language Model. It's the thing that reads records and writes an answer in English. vLLM is a program that runs an LLM on a GPU you control and answers requests over HTTP, the way a web server answers requests for pages. Part 8 uses it so the words inside your tickets never leave a machine you rent. A token is how a model counts text. It's roughly four characters, so about three quarters of a word. It matters because a model can only read so many tokens at once. That limit forces every decision later. A chunk is one piece of text, cut to a size worth storing and retrieving. It can be a whole ticket, or one field of it. An embedding is a list of numbers standing for the meaning of a chunk. Two chunks that mean similar things get similar numbers. That lets a computer find text by meaning instead of by exact words. A vector index is a store of embeddings, built so you can ask "what is closest in meaning to this?" and get an answer quickly. The sys_id is ServiceNow's own identifier for a record: a 32 character string it generates and never shows you unless you ask. It isn't INC0010001. That's the number a human reads, and the sys_id is what every reference between two records actually stores. From Part 4 onwards, this is the difference between a link that works and a blank field that never errors. Retrieval is choosing which records to show the model. The whole book is about this one concept. RAG stands for Retrieval Augmented Generation. Find the relevant records, put them in front of the model, and let it answer from them. GraphRAG is the same idea where a graph decides what is relevant. Those seventeen terms aren't seventeen separate facts. They're three short chains, and each one is easier to hold as a picture than as a list. Let's see how they fit together visually in the following diagrams: Three words, one inside the other. The CMDB is the list of everything you own. One line on that list is a CI, and pg0711 above is one. The sys_id is the 32 character name ServiceNow generates for that line and never shows you unless you ask. From Part 4 onwards the sys_id is the difference between a link that works and a blank field that never errors. Each word here is made of the one before it. One circle is a node. A named arrow between two of them is a relationship. Enough of those together is a graph. A fact stored on a node, like the name hanging off the first circle, is a property, and relationships carry properties too. Cypher is the language you use to ask the finished graph a question. The line in the band is a real one. It says follow SUPPORTS as far as it goes, and hand back everything you reach. Here are five of those words again on one real record out of this book's own data, so you have seen each one on a thing rather than in a sentence. Five words, on one real record. The box is a node. The chip is its label, which is the graph's own name for what kind of thing this is. LinuxServer is the label Part 7 applies, not the ServiceNow class the row arrived under. Each line inside the box is a property. The arrow is a relationship, which is named and has a direction. And the arrow carries properties of its own. The other seven words are one journey a piece of text takes. A token is how the model counts that text, roughly four characters. A chunk is one piece of it, cut to a size worth storing. An embedding is that chunk written as a list of numbers. Two pieces that mean similar things get similar numbers. A vector index holds those numbers so you can ask what is closest. Retrieval is the narrowing, choosing which few pieces the model actually sees. The LLM reads them and writes the answer, and the whole of the last stretch is what people mean by RAG. Three more belong to Part 10, and this part already uses them. Recall is the share of the records a correct answer needs that actually came back. 1.00 is every one of them. 0.00 is none. An arm is one retrieval method, measured against the others. A drug trial has arms, and so does this comparison. Part 10 scores eight. A holdout is a question kept back while the system is being designed. It tests the finished thing, rather than being the thing the design was tuned against. The Route, Part by Part The introduction said what you'll have at the end. This is the route to it. There are ten parts after this one, and each finishes something you can check on your own screen before the next one starts. One thing to expect before you start: the scoreboard at the end doesn't crown a winner, and section 111 explains why that's the useful result rather than a disappointing one. Here's the whole route on one page. Every part is safe to stop after, so this is a weekend project you can put down. Part What you do What you have when it is done Time (Estimated) 1 Create three free accounts and put every key in one file Credentials that work, proved with a 200 40 min, plus one wait 2 Set up Python and clone the code One script that connects to everything and prints ok 15 min 3 Look at the dataset before loading it The row counts you'll check every later number against 15 min 4 Load the estate into ServiceNow 11,891 items and 68,900 tickets in a real instance 60 min, mostly waiting 5 Read it back out with Python Records in memory, with the field traps handled 40 min 6 Decide what the graph should look like A model you can defend, drawn before any code 60 min reading 7 Load the graph into Neo4j A graph you can walk, checked four ways 30 min 8 Rent one GPU and serve two models A language model answering on hardware you control 45 min, billing 9 Build five ways to retrieve Five retrievers, which Part 10 scores alongside three plain baselines 90 min 10 Score all eight against frozen questions A measured table, and an honest account of where every arm failed 60 min There are two things this book won't do. It won't tell you graphs are always better, because Part 10 measures a question where they are not. And it won't ask you for a payment card until Part 8, which is the only part that costs anything. 2. Why This is Hard in ServiceNow Today Section 1 ended with twenty minutes, a notepad, and a list you can't be sure of. It would be easy to blame ServiceNow for that, and it would be wrong. The twenty minutes aren't a bug, a missing feature, or somebody's failure to fill a field in. They fall out of one design decision at the centre of the CMDB, and that decision is the right one for almost everything else the platform does. This section is what that decision is, and why it costs you twenty minutes at 02:10. A CMDB stores each fact as its own row. The payments service is one row. The application is another row. The sentence "the payments service depends on this application" is a third row, in a table called cmdb_rel_ci, holding a parent, a child, and a type. The design is a good one. It means any two items can be connected without changing the shape of the database. The cost of that design appears when you ask a question whose parts live in more than one row. The rest of this section rests on one term, so take that first. A join is how a relational database answers a question like that. You tell it: take this row, find the row its child column points at, and hand me both together. Writing one join is ordinary work. The trouble starts when you don't know how many you need. Count them for the 02:10 question. "What depends on the payments service?" is one row and no join at all. "What depends on what depends on it?" needs one join, because the first row's child has to become the second row's parent. Three deep needs two joins. And "everything that breaks if this breaks" needs a number of joins nobody can write down in advance. The chain stops when it stops, and the only way to learn where is to walk it. When a table joins back to itself like this, once per step, it's called a self join. Each extra step is another one somebody writes by hand. Count the tiles. One hop needs no join at all. Two hops needs one, three hops needs two, and each of those is a line somebody types. The fourth column is the real question and it has no top. The number of joins is whatever the chain turns out to be. That's the problem, and it's not that the query would be slow. Nobody can even write it until they have already walked the chain by hand, which is the twenty minutes with the notepad. The slab underneath is the graph version: one query, and it doesn't change when the chain does. Here's that three hop column written out. This is SQL, and it's correct, and it runs: SELECT c.child FROM cmdb_rel_ci a JOIN cmdb_rel_ci b ON b.parent = a.child JOIN cmdb_rel_ci c ON c.parent = b.child WHERE a.parent = :item Read the two JOIN lines and you can see the table being joined back to itself, once per hop. Three hops, two joins, and a fourth hop would need a third. Here's the same question in Cypher, which is the language Neo4j takes: MATCH (a)<-[:SUPPORTS*]-(b) WHERE a.key = $item RETURN b The * is the whole difference: it means follow this relationship as far as it goes. Nothing in that line says how deep. So nothing has to change when the answer is four levels down instead of three. Here is the objection a reader who knows SQL is already making, and it's a fair one. Standard SQL can walk a chain of unknown length. WITH RECURSIVE has been in the standard since SQL:1999, and Postgres, MySQL, Oracle and SQL Server all have it. One statement does the whole open-ended walk: WITH RECURSIVE impacted AS ( SELECT parent FROM cmdb_rel_ci WHERE child = :start AND type IN ('Depends on::Used by', 'Runs on::Runs', 'Hosted on::Hosts') UNION SELECT r.parent FROM cmdb_rel_ci r JOIN impacted i ON r.child = i.parent WHERE r.type IN ('Depends on::Used by', 'Runs on::Runs', 'Hosted on::Hosts') ) SELECT DISTINCT parent FROM impacted; So "a relational database can't answer this" would be false, and I'm not going to write it. It can. The true claim is a narrower one, and it has three parts. The first is reading it. Put that statement beside the two lines of Cypher above. Both are correct. Only one of them gets typed from memory at 02:10 by somebody who has never typed it before. The second is the row shape. cmdb_rel_ci keeps the relationship type as a string in a column. So the type filter is written twice, once in the first half and once in the recursive half. Change your mind about which types carry impact and you edit both halves. Edit one and the query still runs. The third is direction, and Part 6 section 55 is the whole story. The type name says which end is which, so Hosted on::Hosts means the parent is hosted on the child. In a graph that decision is made once, when Part 7 loads the edge and names it. In SQL it's made again inside every recursive query anybody writes. I got it wrong once and 55.9% of my edges pointed backwards. Nothing errored and every count was right. That query can be written, and on the 28,694 relationship rows in that same estate it will work. Writing it was never the expensive part. Reading it, checking it, and getting its direction right at 02:10 is. The data is all there. Getting it out in one answer is the problem. 2b. What ServiceNow Already Gives You, and Why This Book Exists Anyway Before going further I have to be straight with you, because a CMDB owner reading section 1 will already be objecting. The objection is that ServiceNow is not the empty box section 1 made it sound like, and that objection is correct. The platform ships real tools for walking the CMDB, and some of them are very good. So here's the real split: what those tools already answer on one side, and what this book starts from on the other. Structural questions go left. Anything that needs the ticket text goes right. By ticket text I mean the words a person typed into an incident rather than picked from a dropdown: its short description, its description, and its work notes. Those fields are free text. Nothing in them is categorised, so no filter and no report can reach what they say. On the left, Dependency Views opens from one configuration item's own record and draws the map of what that item connects to. The depth is how many relationship hops out it follows: depth 1 is the item's immediate neighbours, depth 3 is everything within three hops of it. CI Impact Explorer and the Impact Analysis API compute what breaks when something breaks. CMDB Query Builder writes multi-hop queries with no code. CMDB Health measures staleness, completeness and correctness with dashboards. Service Mapping keeps application service maps current on its own. All five are real, supported, and better maintained than anything in this repository. The right side is the list of things none of those five tools does, and it is what this book builds. A question typed in English rather than into a form or a filter. The free text of sixty thousand tickets, where a symptom nobody categorised sits in the words an engineer used. One walk that crosses incidents, changes, problems, and knowledge together with the infrastructure. Evidence handed to a model so the answer arrives as a sentence. And a measurement of which retrieval strategy actually returned the right records. ServiceNow doesn't make you click through records one at a time. It ships tools for exactly the walk I just described: Dependency Views draws the map from a configuration item's form, to a depth you choose, filtered by relationship type. CI Impact Explorer and the Impact Analysis API compute what breaks when something breaks. CMDB Query Builder writes multi-hop graph queries with no code at all. CMDB Health already measures staleness, completeness and correctness, with dashboards. With ITOM licensed, Service Mapping keeps application service maps current on its own. If your question is "what depends on this item", use those. They're built in, they're supported, and they are better maintained than anything you'll write. Two more ServiceNow products belong on that list, and these two compete with this book directly: ServiceNow AI Search is the platform's own search engine. It reads a question phrased the way a person would phrase it. It ranks results across the tables it indexes, and can hand back an answer card rather than a list of links. It comes with the platform rather than as a separate purchase. It does have to be configured and indexed first. Now Assist is ServiceNow's generative AI layer. It summarises a long incident and drafts a resolution note. It answers a question in English from knowledge articles and the records nearby. So "you can't ask ServiceNow a question in English" is not a sentence I'm willing to write. Now Assist does exactly that, and the people who built the tables built it. There's also a privacy point I should concede here rather than bury. The ticket text already lives in ServiceNow. A ServiceNow product reading it changes nothing about who holds it, which is not true of a hosted API from somebody else. What survives is narrower, and it's about price and about proof. Now Assist is a paid add-on, licensed on top of your platform subscription. It isn't on the free developer instance this book uses. Everything here before Part 8 costs nothing. If your employer already pays for Now Assist, use it. That is a straight recommendation and not a hedge. So here's the straightforward case for this book. The five tools above answer structural questions about the CMDB. None of those five does any of this: Takes a question typed in English. Searches the free text of sixty thousand tickets for a symptom nobody categorised. Puts incidents, changes, problems and knowledge in one walk with the infrastructure. Hands the evidence to a language model, so the answer comes back as a sentence. Lets you measure which retrieval strategy actually found the right records. That last bullet holds for AI Search and Now Assist too. It's what most of this book is really about. Neither of them publishes a number you can check against your own estate. The retrieval happens inside the product, there's no answer key, and nothing reports which strategy returned the right records. If the tools above are all you need, close the tab and open Dependency Views. If you want that measurement, keep reading. 3. Why Plain Search Doesn't Solve it The clear modern answer is to point a search engine at the data. Put every record into a vector index, ask your question in English, and let the model read what comes back. This is what most people mean by RAG. The five kinds of question are five different movements through the data. Landing on a record you can name is one motion. Walking from it is a second. Gathering and counting is a third Putting things in order is a fourth. The fifth is landing on a record you can't name, by meaning rather than by spelling. A keyword index can only do the first. It scores 1.00 on landing, 0.50 on walking and 0.00 on the other three, which the chart below draws. Keyword search is perfect when you can name the record you want. It scores zero when the answer has to be counted, ordered in time, or found by meaning. Every score here is recall inside a 3,000 token budget. The counts behind each row are small and the table below prints them. The three zeros aren't a keyword problem. Semantic search, a hybrid of the two, and no retrieval at all scored 0.00 on the same three kinds. Both halves hold the same three rows, which is what the equals sign means. Each of them is one row of cmdb_rel_ci, and there are 28,694 of those in this dataset. A row names two things and the way they relate, and that's all it does. Nothing in the table joins row one to row three. On the right the identical three rows are drawn end to end, and row one now reaches row three. Following the arrows is the only thing a graph adds. For some questions this works very well. For this question it doesn't, and it's worth being precise about why. Search finds records that look like your question. That's all it does. Ask it about the payments service. It finds every record with the word payments in it, ranked by how closely the wording matches. Those records are genuinely relevant. But the thing you need isn't worded like your question at all. The storage array under your payments service doesn't have the word payments anywhere on it. It's called san-eu-west-01. It's a storage server in the eu-west region, owned by the platform team. Every word on its record is about storage. Nothing about its text resembles what you typed. Search can't find it, because a single search has no way to follow a chain from one record to another. That word "single" is doing real work, and I'm not going to hide behind it. An agent can do this without a graph. It issues one query, reads the answer, spots pg0711 in the text, then issues a second query for that. It reaches the storage array in the end. Multi-step retrieval is a real technique and it works. It's slower and it costs a model call per hop. It's also only as reliable as the model's decision about what to search for next. A traversal is one query with a known answer. But "search can't do this" would be false, and the true claim is that a single-shot search can't. I measured this rather than assuming it. The question set was written and locked before any search code existed. Part 9 cuts those records into 82,296 searchable pieces. Here's keyword search over all of them: kind of question keyword search finds questions behind it look up a record you can name 1.00 3 follow a chain of dependencies 0.50 2 find something by meaning 0.00 1 count or rank something 0.00 2 compare things in time 0.00 2 Those counts are small, and they're printed for a reason. The question set is thirty nine questions, written and hashed before any retrieval code existed, so nothing in the book could be tuned to them. Part 10 section 106 lists all thirty nine and shows how they were frozen. Twenty one of them carry a mechanical answer, meaning somebody can write down in advance which records a correct answer needs, rather than having to read the answer and judge it. Ten of those twenty one have an answer key small enough to score recall against, and recall is the share of the records a correct answer needs that actually came back. Those ten are the only questions that get a number in the recall column of Part 10's results table in section 111. That is what the counts in the last column above are drawn from. A cell resting on two questions isn't a law of nature. Read the whole table as a direction, not a measurement of the universe. Part 10 gives the full set and the statistics. Compare the first row with the last three. Again, keyword search is perfect when you can name the thing you want. It scores zero when the answer has to be counted, ordered in time, or found by meaning rather than by words. The chain row is the interesting one, and it needs a warning label. Half isn't a failure and it isn't a success. The two questions behind it are graded to different depths. One is scored against the whole chain, sixteen items reaching the storage array, and keyword search scored zero on it. The other is scored against one hop only, four items, and keyword search got all four. So the 0.50 is a full-depth miss beside a one-hop hit. Part 10 section 108b prints both answer keys. One thing about this dataset changes how you should read that table, so you are entitled to know it now. The items in it are named lnx2419, pg0711, app0958. That is an infrastructure naming scheme, where nothing in a name tells you what sits above or below it. That matters for the comparison. Say a service were called payments-app and its database payments-db. A plain text search could then recover the whole stack from the names alone. The graph would look clever for finding what the spelling had already given away. Real estates don't name a database after the service that uses it, because different people name different things at different times. So the published dataset uses names that carry no structure. The comparison has to be won by the graph, not by the spelling. Part 10 section 117b returns to this and says how much of the result the naming decides. But judge that for yourself rather than take it from me. The result in the table above depends on it. With stack-correlated names, keyword search does much better at following a chain. With realistic names, it doesn't. Part 10 repeats this table with a vector index, a hybrid of the two, and three arms that walk the graph. It reports which arm won. It isn't the one this book is named after. The numbers above are one row of a longer table, published so you can check them rather than take my word. 4. The Four Questions This Book Answers Everything here is built to answer four questions. They're the four that come up in a real incident, and each one needs something a search index can't do. Here they are, in the order the rest of the book takes them. Blast radius: This item is broken. What else stops working? Needs a chain followed upward, however long the chain turns out to be. Change correlation. Something broke at 02:10. What changed near it recently? Needs the graph to decide what "near it" means, and time to decide what "recently" means. Shared root cause. Three incidents are open on three different systems. Do they share something underneath? Needs three chains followed downward until they meet, or a clear answer that they never do. Finding the past fix. This looks familiar. Has it happened before, and what worked? This one genuinely needs search, because the symptom is written in free text and no two people describe it the same way. Two of these four are scored in Part 10 and two aren't. Blast radius is scored in section 111 as a multi-hop question, and finding the past fix as a lookup. The other two carry no mechanical answer key, so neither can sit in a recall column at all. Change correlation names an incident that sits on a staging host rather than on the payments service. That's reported rather than rewritten, because the question was frozen before the data existed. Shared root cause is a judgement about three open tickets, and it was frozen as a holdout besides. It never fed the comparison, and that's what stops a comparison being tuned to the questions it answers. Ten of the thirty nine questions have an answer key small enough to score recall against, so ten is the number behind every recall figure in Part 10. Section 116 says what a comparison resting on ten questions does and does not let you claim. That fourth question matters more than it looks. It's the one a graph is worst at and a text index is best at. It's in the list on purpose. A book where the graph wins every question isn't a comparison. It's a sales page, and you shouldn't trust one. 5. When You Shouldn't Build This I would rather you stop reading now than build something that doesn't help you. The second of these three panels is what the middle line on the next chart means. The first answer lists everything that breaks. The second stops early, and nothing on it says so. It has no error, no gap, and no marker. It's drawn in the same ink as the correct one on purpose. The third answer is empty, which is the only one a person notices. That's why a lightly stale CMDB is more dangerous than an obviously broken one. As the dependency data degrades, wrong answers don't announce themselves. I removed dependency edges on purpose and re-asked the opening question. The sample is 242 production services, with 25 random draws at each level of damage. The dangerous line is the middle one. Lose one edge in twenty and a quarter of the answers return short. It peaks near 30% damage and then falls, because the answers start returning empty instead, and an empty answer makes somebody check. This dataset's own staleness is 17.89% of dependency edges over a year old, and yours is the number that matters. So here are three times this is the wrong tool. One is looking up a record you can already name. If you know the ticket number, open the ticket. A graph adds nothing and costs real money. Another is wanting to know whether something is working right now. A CMDB records how things are connected. It doesn't record whether they're running. That's monitoring, and this isn't monitoring. The third one matters most: relationship data you know to be wrong. Everything here rests on the dependency rows in your CMDB being roughly correct. If your organisation hasn't maintained them, a graph will answer confidently and wrongly. That's worse than answering slowly and being right. Before you build anything, check. Part 6 shows you how to measure what fraction of your dependency data hasn't been confirmed in over a year. In the dataset used here, that number is 17.89%. Don't carry that figure to your own estate. It's a property of a generated one. Section 65 shows the shape behind it is arithmetic, not a fact about CMDBs. The number that matters is yours. Telling you that without telling you what it costs would be useless. So I removed dependency edges on purpose and re-asked the opening question. The sample is every production service with a blast radius of three or more, 242 of them. Each row is 25 random draws of which edges go missing: Edges missing Exactly right Short and plausible Empty 0% 100% 0% 0% 5% 72% 25% 3% 10% 52% 41% 7% 20% 28% 57% 15% 30% 14% 61% 24% 50% 4% 54% 42% Look at the 5% row. Lose one edge in twenty, and a quarter of your blast radius answers are quietly wrong. Not empty. Not an error. Shorter, and shorter looks exactly like correct. Now follow the last column down. Empty answers only become common once the damage is severe, and an empty answer is the one a person notices. The short-and-plausible column peaks near 30% damage and then falls, because the answers start coming back empty instead. That fall holds in all 25 draws. The position of the peak is softer, landing on 30% in 20 of them. So the uncomfortable finding is this: a lightly stale CMDB is more dangerous than an obviously broken one. At 5% damage you get a quarter of your answers wrong and almost nothing that looks like a problem. If your data is worse than lightly stale, fix your CMDB first. Nothing in this book will save you from bad data. A confident wrong answer at 02:10 is the worst outcome of all. 6. What You'll Build By the end you'll have your own ServiceNow data standing up as a graph. You'll also have a way to search it, and a scoreboard that says which search found the right records. Both are called GraphRAG and the difference is where the edges came from. On the left in the image above, a model reads the documents, pulls out the entities, guesses the relations, and a graph nobody wrote appears. Nobody can put a number under it, which is what the empty slots mean. On the right, the edges were written down before anybody asked a question. In a real estate, people wrote them, and in this published dataset, a seeded script did. Part 3 section 29 is blunt about which parts are which. The counts are read straight out of the dataset as the picture is drawn. The job here is moving that graph without breaking it, then hanging the ticket text off it. GraphRAG means two different things in public, so here's which one this is. Most published GraphRAG work extracts a graph out of unstructured text: read the documents, pull out entities and relations, build a graph nobody wrote down. That isn't this. The graph here is already written down, in the CMDB, by the people who run the estate. This book's job is to move it without breaking it, then measure whether it helps. The ticket text hangs off that graph as chunks. If you came for entity extraction from prose, this isn't the right resource, and section 118 says where that would go. There are four pieces we're working with here, and the grouping is the point. The first three are free and need no payment card. They're a personal ServiceNow instance, your items standing up as a graph in Neo4j, and eight retrieval strategies. Those eight are five designs and three baselines. The fourth is explained in Part 8. Both models run on one rented GPU, so the ticket text never leaves a machine you control. It's the only part that costs anything. Budget $5 for it: a clean run is $1.24 and the work behind this book billed $4.19. The four pieces are: A real ServiceNow instance, read through its own API. Real tables and real field behaviour, including the parts that behave in ways the documentation doesn't mention. A real Neo4j database, holding your configuration items and the relationships between them as a graph you can walk. Eight retrieval strategies, measured on this corpus. Two of them are controls that let the comparison fail. Part 10 reports which won and which lost, on its face. A GPU you rent by the hour, running both models on one card. For CMDB text, whether it left your control is usually what decides whether the project is allowed. That's Part 8, and it's the only part that costs money. The point is this: This isn't a demonstration that graphs are good. It's a measurement of when they are and when they aren't. The code and the data are one clone. Every script, the question set, the gold answers, and the scoring harness are in one repository. So is the estate this book measures: git clone https://github.com/ronidas39/servicenow-graphrag.git cd servicenow-graphrag ls You should see seven directories and a requirements file: dataset/ the files you will load into ServiceNow generator/ the loaders, for ServiceNow and for Neo4j gpu/ launch, measure and teardown for Part 8 questions/ the frozen question set and the gold answers results/ the scores Part 10 publishes, so you can check them retrieval/ chunking, the retrieval arms, the scoring tests/ the tests that prove the above requirements.txt Don't install anything yet. Part 2 section 24 builds a virtual environment first, and section 25 installs into it. Installing these packages into your system Python now is the one step in this book that's genuinely awkward to undo. dataset/ holds the records: 11,891 configuration items, 28,694 dependency rows, 60,000 incidents with their work notes, 8,000 changes, 900 problems, and 301 knowledge articles. They're generated, not scraped. Part 6 section 58 is blunt about which parts are realistic and which are a setting in the generator. A real CMDB is somebody's confidential estate, so a book built on one is a book you can't reproduce. results/ holds the numbers Part 10 publishes, including the per-question scores, so you can check the tables rather than believe them. 6b. The Other GraphRAG, and the Work This One Isn't Section 6 above said this book moves a graph that already exists. The published research mostly does the opposite. If you've read any of it, you should know where the line falls before you read on. Microsoft's GraphRAG is the one most people mean. "From Local to Global: A Graph RAG Approach to Query-Focused Summarization" (Edge and others, arXiv 2404.16130) reads a corpus with a model. It extracts an entity graph, finds communities in that graph, and pre-writes a summary of each one. Ask it a broad question and it answers from the summaries rather than from the documents. That's a different problem from this one. It's for corpora with no structure, and its hard part is building a trustworthy graph out of prose. Two more are worth knowing, and both are about retrieval rather than summarising. HippoRAG (arXiv 2405.14831) builds an entity graph and runs Personalised PageRank over it