Showing posts with label JSON. Show all posts
Showing posts with label JSON. Show all posts

16 October 2024

Display JSON in APEX using APEX_MARKDOWN and JSON_TRANSFORM #JoelKallmanDay

Markdown is a plain text format for writing structured documents, and in APEX there is a built-in package that can transform Markdown notation into HTML.
With Markdown it is also possible to show pieces of code or JSON documents in a fixed size font. Having a fixed font to display JSON documents in APEX is easier to read than a non-fixed font.
The idea is to add Markdown to the JSON document and use the APEX_MARKDOWN package to convert the whole string in HTML.
Something like:

  ```json
  {"this":"that"}
  
The first line in the Markdown starts with three ticks (`) followed by "json", then the JSON document start on a new line.

To setup the example, use the following script to create a table and add some sample data. The sample data consist of three different JSON documents. Note that the first document includes a PDF file, the Base64 value representind the PDF has been shortened for brevity. In reality the Base54 value is enormous, but there is a solution for that in this post.

create table t 
(outgoing_message json) -- Oracle Database 23ai introduced the JSON datatype
/
insert into t (outgoing_message) 
values (
  '{
    "orderNumber" : "APX000328573",
    "invoiceNumber" : 2023300028,
    "salesChannel" : "Amazon",
    "file" :
    {
      "mime" : "application/pdf",
      "data" : "255044462D312E340A25C3A4C3BCC3B6C39F0A322030206F626A0A3C3C2F4C656E6774682033203020522F46696C7465722F466C6"
    }
  }'
);

insert into t (outgoing_message) 
values (
  '{
    "status" : "success",
    "message" : "The order update was successfully received."
  }
  '
);

insert into t (outgoing_message) 
values (
  '{
    "orderNumber" : "APX000328573",
    "timeStamp" : "2023-04-13T14:49:50.453269Z",
    "salesChannel" : "Amazon",
    "financialStatus" : "paid",
    "logisticStatus" : "ready for production",
    "payments" :
    [
      {
        "provider" : "Mollie",
        "date" : "2023-04-13T16:49:50",
        "amount" : 569.95,
        "currency" : "EUR"
      }
    ]
  }');
commit;

Displaying the complete Base64 string representing the PDF document does not add any value for the user looking at the JSON document and is most likey irrelevant.
Using the JSON_TRANSFORM function it is trivial to replace the comple Base64 string into something that is good enough for the user, in this example we will replace it with "<Base64Encoded PDF Document>".
Depending on the version of the database that you're using, the syntax has changed slightly from Oracle Database 19c to Oracle Database 23ai, use the following snippet:

-- Oracle Database 19c 
select json_transform (ann.outgoing_message
          ,set '$.file.data' = '<Base64Encoded PDF Document>'  ignore on missing
          returning clob 
         ) as outgoing_message
  from t ann


-- Oracle Database 23ai
select json_transform (ann.outgoing_message
          ,nested path '$' (
             set '@.file.data' = '<Base64Encoded PDF Document>' ignore on missing
            )
         ) as outgoing_message
  from t ann
;

The next step is to wrap the resulting JSON document with the Markdown tags and a new line break:

apex_markdown.to_html ('```json'||chr(10)||
This results in the following query and result:
   select json_transform (ann.outgoing_message
            ,nested path '$' (
               set '@.file.data' = '<Base64Encoded PDF Document>' ignore on missing
            )
         ) as outgoing_message
  from t ann;
  
  
<pre><code class="language-json">{
  "orderNumber" : "APX000328573",
  "invoiceNumber" : 2023300028,
  "salesChannel" : "Amazon",
  "file" :
  {
    "mime" : "application/pdf",
    "data" : "<Base64Encoded PDF Document>"
  }
}
</code></pre>

Links

02 March 2022

Modify a JSON structure with a single Update

Recently there was a question on the Oracle Community Forum related to JSON, more specifically how to change the structure of it. The link to the Question is at the bottom of this post.
The starting point is the following JSON:

  {
   "tktKey": "123456",
   "attlist1": [
      {
         "attkey": "123",
         "attval": "abc"
      },
      {
         "attkey": "456",
         "attval": "def"
      }
   ],
   "attlist2": [
      {
         "attkey": "098",
         "attval": "xyz"
      }
   ]
}
  
The task at hand was to combine both attlist1 and attlist2 into a single attlist.
To do this the JSON object that is contained in attlist2 need to be extracted and place in attlist1. After that the element attlist2 can be removed from the structure. When this is done, there is an element with attlist1 with three JSON objects in it. Finally the element attlist1 needs to be renamed to attlist.
All of these actions can be done with a single SQL statement using JSON_TRANSFORM.

Let's first create a table to hold the original JSON
   SQL> create table js
  2  (str clob 
  3  )
  4* /

Table JS created.
   
And insert the JSON Object in there.
   SQL> insert into js values ('{ "tktKey": "123456", "attlist1": [ { "attkey": "123",   "attval": "abc"  },  { "attkey": "456","attval": "def"  } ],  "attlist2": [{"attkey": "098", "attval": "xyz"  } ]}')
  2* /

1 row inserted.
   
Here comes the magic moment:
   SQL> update js
  2     set str= json_transform
  3                (js.str
  4                ,append '$.attlist1'
  5                    = json_query (js.str, '$.attlist2[*]')
  6                ,remove '$.attlist2'
  7                ,rename '$.attlist1' ='attlist')
  8* /

1 row updated.
   
With JSON_QUERY (line 5) I extract the JSON Object from attlist2, which is added (APPEND, line 4) to attlist1.
Attlist2 is removed from the JSON object (line 6) and finally on line 7 attlist1 is renamed to attlist.

And to view the results, in a nice readable format:
SQL> select json_query (j.str, '$' returning clob pretty) as result
  2    from js j
  3* /

RESULT 
_______ 
{
  "tktKey" : "123456",
  "attlist" :
  [
    {
      "attkey" : "123",
      "attval" : "abc"
    },
    {
      "attkey" : "456",
      "attval" : "def"
    },
    {
      "attkey" : "098",
      "attval" : "xyz"
    }
  ]
}    

The more I use the JSON functionality that the Oracle database has to offer, the more I like them!


Links

26 November 2019

Human readable JSON, stored in BLOB

Currently Oracle (we're using Oracle 18c at the moment) is still recommending to store your JSON document in a BLOB column. More efficient, no character conversions and probably some more advantages.

When there is a need to look at the JSON document, having it as a BLOB is not very useful.

An easy way to make the JSON Blob readable is to use the following query:

select json_query (i.json_payload, '$' returning clob pretty)
  from imports i
Omitting "Pretty" will output the JSON object without formatting. Including Pretty makes it look... well... pretty.

09 January 2018

JSON_TABLE and the Top 2000

Oracle database 12c provides native JSON parsing. It is relatively easy to go from data in a JSON format to a relational representation.
In this example I will use data from the Top 2000. Every year in The Netherlands a playlist is produced by popular vote and fully broadcast between Christmas and the last day of the year, with the number one being played just before the New Year. You can see the complete list of 2017 on the official NPO website.
The Top 2000 list that I'll use for this example is the one from 2017.

The data from the JSON file looks like this:

Only the first part of the JSON file is shown, with the first two songs.
At the highest level there are three name-value pairs ("success", "message" and "messages") and an array named "data".
The "data" array contains another array with JSON objects containing information about the songs.
Each of these JSON objects contain name-value pairs, with very short none descriptive names, like "s" and "a". You might guess what these names would mean.
Even though the "data" attribute is a nested array, at the top level it is only one level deep.
The complete JSON-file can be downloaded by following this link.

At first I want to get the JSON file into the database, before I extract the values that I need.
First create the table and a check constraint to make sure that is JSON.

create table temp2000 
(complete_list clob);

alter table temp2000
add constraint list_is_json check (complete_list is json)
/
This table will hold the complete JSON file containing the Top2000. The check constraint on the column will verify that only correctly formatted JSON is allowed.
Now add the JSON-file to the table, the easiest way to do this is by adding a row using SQL Developer and copy-paste the complete JSON.

What I want to extract from the complete JSON file is the information about the artist, songtitle, release year, current position in the list, as well as the position in the list from last year.
My guess would be that the "a", "s", "yr", "pos", "prv" are the attributes that I need.
With a CTAS (Create Table as SELECT) and the JSON_TABLE operator I can transform the JSON to relational representation.

create table top2000
as
select songtitle
      ,artist
      ,release_year
      ,curr_position
      ,prev_position
  from temp2000 t
      ,json_table (t.complete_list format json, '$.data[0][*]'
         columns (
           songtitle     varchar2(150) path '$.s'
          ,artist        varchar2(150) path '$.a'
          ,release_year  number path '$.yr'
          ,curr_position number path '$.pos'
          ,prev_position number path '$.prv'
         )
      )
/
Because the song information is contained in the "data" array, and only in the nested array, I need to address that array as follows:
'$.data[0][*]'
Zero (in square brackets) representing the first array in the "data" attribute, and a wildcard (the asterisk) to address the other objects in the nested array.

To help with the discovery of the structure (and the paths to the values) of the JSON, Oracle Database 12c Release 2 introduced the JSON_DATAGUIDE function. Unfortunatelly I don't have Release 2 at my disposal right now, so I leave that for a later time.

Links