Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
P
pdf-miner
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Qin Kaijie
pdf-miner
Commits
58c191e7
Commit
58c191e7
authored
Apr 08, 2024
by
赵小蒙
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
将s3的init配置转换成json配置,并保存到home目录下
parent
34bde6d8
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
57 additions
and
0 deletions
+57
-0
config_init_to_json.py
magic_pdf/spark/config_init_to_json.py
+57
-0
No files found.
magic_pdf/spark/config_init_to_json.py
0 → 100644
View file @
58c191e7
from
loguru
import
logger
import
json
import
os
from
magic_pdf.config
import
s3_buckets
,
s3_clusters
,
s3_users
def
get_bucket_configs_dict
(
buckets
,
clusters
,
users
):
bucket_configs
=
{}
for
s3_bucket
in
buckets
.
items
():
bucket_name
=
s3_bucket
[
0
]
bucket_config
=
s3_bucket
[
1
]
cluster
,
user
=
bucket_config
cluster_config
=
clusters
[
cluster
]
endpoint_key
=
"outside"
endpoints
=
cluster_config
[
endpoint_key
]
endpoint
=
endpoints
[
0
]
user_config
=
users
[
user
]
# logger.info(bucket_name)
# logger.info(endpoint)
# logger.info(user_config)
bucket_config
=
{
"endpoint"
:
endpoint
,
"ak"
:
user_config
[
"ak"
],
"sk"
:
user_config
[
"sk"
],
}
bucket_configs
[
bucket_name
]
=
bucket_config
return
bucket_configs
def
write_json_to_home
(
my_dict
):
# Convert dictionary to JSON
json_data
=
json
.
dumps
(
my_dict
,
indent
=
4
,
ensure_ascii
=
False
)
# Determine the home directory path based on the operating system
if
os
.
name
==
"posix"
:
# Linux or macOS
home_dir
=
os
.
path
.
expanduser
(
"~"
)
elif
os
.
name
==
"nt"
:
# Windows
home_dir
=
os
.
path
.
expandvars
(
"
%
USERPROFILE
%
"
)
else
:
raise
Exception
(
"Unsupported operating system"
)
# Define the output file path
output_file
=
os
.
path
.
join
(
home_dir
,
"magic_pdf_config.json"
)
# Write JSON data to the output file
with
open
(
output_file
,
"w"
)
as
f
:
f
.
write
(
json_data
)
# Print a success message
print
(
f
"Dictionary converted to JSON and saved to {output_file}"
)
if
__name__
==
'__main__'
:
bucket_configs
=
get_bucket_configs_dict
(
s3_buckets
,
s3_clusters
,
s3_users
)
logger
.
info
(
bucket_configs
)
write_json_to_home
(
bucket_configs
)
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment