{"id":156,"date":"2024-09-28T20:11:00","date_gmt":"2024-09-28T18:11:00","guid":{"rendered":"https:\/\/codeswarm.io\/?p=156"},"modified":"2026-07-19T20:17:27","modified_gmt":"2026-07-19T18:17:27","slug":"apache-iceberg-and-time-travel-queries-with-scala-spark","status":"publish","type":"post","link":"https:\/\/codeswarm.io\/?p=156","title":{"rendered":"Apache Iceberg and time travel queries with Scala\/Spark"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Iceberg provides <strong>time travel<\/strong> functionality, allowing you to query the state of a table at a previous point in time. You can query the table as it existed at a specific snapshot or as of a certain timestamp.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Example: querying an Iceberg table at a specific snapshot<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Each time you make changes (like insert or update) to an Iceberg table, a snapshot is created. You can query data from a particular snapshot.<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"scala\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">def timeTravelBySnapshot(spark: SparkSession, snapshotId: Long): Unit = {\n  val timeTravelDF = spark.read\n    .option(\"snapshot-id\", snapshotId)\n    .table(\"spark_catalog.default.iceberg_table\")\n  \n  timeTravelDF.show()\n  println(s\"Data retrieved for snapshot: $snapshotId\")\n}<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">We can find the snapshot ID using:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"scala\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">val historyDF = spark.sql(\"SELECT * FROM spark_catalog.default.iceberg_table.history\")\nhistoryDF.show()<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Once we have the snapshot ID, pass it to <code>timeTravelBySnapshot()<\/code> to query the data as of that snapshot.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Example: querying an Iceberg table as of a timestamp<\/strong><\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"scala\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">def timeTravelByTimestamp(spark: SparkSession, timestamp: String): Unit = {\n  val timeTravelDF = spark.read\n    .option(\"as-of-timestamp\", timestamp)\n    .table(\"spark_catalog.default.iceberg_table\")\n  \n  timeTravelDF.show()\n  println(s\"Data retrieved as of timestamp: $timestamp\")\n}\n\/\/ Example timestamp format: \"2024-10-16T15:00:00.000Z\"\n\/\/ To execute:\ntimeTravelByTimestamp(spark, \"2024-10-16T15:00:00.000Z\")<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">In this way we can to handle time travel queries using Apache Iceberg and Spark.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Iceberg provides time travel functionality, allowing you to query the state of a table at a previous point in time. You can query the table as it existed at a specific snapshot or as of a certain timestamp. Example: querying an Iceberg table at a specific snapshot Each time you make changes (like insert or &hellip; <a href=\"https:\/\/codeswarm.io\/?p=156\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">Apache Iceberg and time travel queries with Scala\/Spark<\/span> <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[24,23],"tags":[21,22,19],"class_list":["post-156","post","type-post","status-publish","format-standard","hentry","category-big-data","category-programming","tag-iceberg","tag-spark","tag-scala"],"_links":{"self":[{"href":"https:\/\/codeswarm.io\/index.php?rest_route=\/wp\/v2\/posts\/156","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/codeswarm.io\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/codeswarm.io\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/codeswarm.io\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/codeswarm.io\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=156"}],"version-history":[{"count":1,"href":"https:\/\/codeswarm.io\/index.php?rest_route=\/wp\/v2\/posts\/156\/revisions"}],"predecessor-version":[{"id":157,"href":"https:\/\/codeswarm.io\/index.php?rest_route=\/wp\/v2\/posts\/156\/revisions\/157"}],"wp:attachment":[{"href":"https:\/\/codeswarm.io\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=156"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/codeswarm.io\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=156"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/codeswarm.io\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=156"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}